Researchers Train Computers to Manipulate Human Emotions with Art

A musician and a computer scientist have created a software system, called "TransProse," that transforms written works into piano concertos. It's all done by training the software to understand how words and sounds create emotional responses in humans.

Here's what A Clockwork Orange sounds like.

In a recently published paper, the two researchers—Hannah Davis and Saif Mohammad—describe how the heart of their system is an algorithm that counts the density of words associated with eight basic emotions: anticipation, anger, joy, fear, disgust, sadness, surprise and trust.

But, "the challenge in composing new music, just as in creating a new story, is the infinite number of choices and possibilities," say Davis and Mohammad. So, they created a series of rules to act as guidelines for the various elements of the music. For instance, music in major key is paired with "positive novels," while the minor key is assigned to "negative novels." Other rules are applied to tempo, scale and dissonance "to make the sequence of notes sound like music as opposed to a cacophonous cascade of sounds."

The final compositions consist of melodies representing the primary and secondary moods in the book's text. Thus, we can listen to a concerto of A Clockwork Orange, which takes its musical cues from "fear" and "sadness," or The Little Prince, steeped in "trust" and "joy."

Davis and Mohammad emphasize that their software represents initial ideas to inspire further development. But, they already foresee various applications, such as creating audio-visual e-books that generate music appropriate to the mood of the text when certain pages are opened. Or TransProse could be integrated into data visualization: imagine a world map where clicking on a particular region plays melodies that capture the emotions of the tweets emanating from there in real time...the music of the Twitterverse.


You can listen to samples of other compositions, including Peter Pan and The Adventures of Sherlock Holmes, on the TransProse website.