When Stevie Wonder wrote "Music is a world within itself, with a language we all understand," the word "all" meant "all people." But that was in 1976. With the rapid advancement of technology, music is gradually becoming a language that can also be understood by computers, so that computers can have their own "opinion" about music and musical styles.
Music is a fairly complex type of data that introduces a challenge for computing machines. Identifying the beat, chords, or musical instruments can provide a computer with some information about a musical piece, but the concept of music is greater than merely its describable components. Even the exact same song can be produced or performed in various different ways, and each version can make a different impact or trigger different emotions. So to allow a computer to "listen," we used as much information as possible about each song, including pieces of information that do not necessarily have a formal definition, but carry information about the music. That was done by first converting the audio into a two-dimensional representation called "spectrogram," basically turning a music analysis problem into an image analysis problem. For instance, the following image is a visualization of Bobby Bland's "dreamer". The horizontal axis is time, the vertical axis is frequency, and the brightness is the volume, so the image actually contains all information about that song.
Then, we computed from each spectrogram almost 3000 numerical values that reflect its texture, shapes, edges, fractals, statistics, polynomial decomposition, and more. These 3000 values were then analyzed by applying algorithms that can identify repetitive patterns typical to a certain group of songs, but are not so typical to other groups of musical pieces. The shared patterns between different musical styles were quantified and visualized.
For instance, if we take several different musicians such that each musician is represented by 100 songs, and ask the computer to automatically identify patterns that are different or similar between these musicians, the computer answers with the following graph. The graph was generated automatically by analyzing the music alone, and the computer was not provided with any other information about any of the songs or musicians.
Shorter paths between two nodes in the graph mean that the two musicians are more similar to each other. So the computer identified that Bob Marley is more similar to Jimmy Cliff than to the Beatles. That is not something we did not know before, but it shows that the computer's "opinion" about the music is pretty much like how a person would sort these musicians.
To analyze the music of a certain artist, we can analyze the similarities between albums, as studio albums are considered the primary musical milestones by which music is analyzed and discussed in historical perspective. Naturally, our starting point was the Beatles, shown in the graph below.
We can see that the albums are sorted in a chronological order, showing how the sound and music of the Beatles progressed gradually from their early albums through their last album, Abbey Road. The computer also identified that Let It Be was recorded before Abbey Road, although it was released after it. Interestingly, the most significant leap is identified between the albums Beatles for Sale and Help.
Queen released about as many albums as the Beatles, but their albums span over a much longer period of time. The computer analysis of the similarity between Queen's albums is the following:
It is interesting that the computer identified a branch starting with the album Hot Space, which is about the time the band changed its musical style to a more 80's-like sound.
The band ABBA seems to be fairly consistent with the way their music changed.
The Bee Gees are a little more complicated case. The band is mostly known for its signature disco style, but in fact the disco era was rather short compared to the time the band was active, spanning over five decades. The computer actually separated between the soft rock, R&B, disco, and synth pop eras of the band.
In the case of Tears for Fears, the album Everybody Loves a Happy Ending is placed by the computer after Seeds of Love and before the albums Elemental and Raoul and the Kings of Spain. That could be because these two albums were created by just one half of the original band members, while Everybody Loves a Happy Ending is an album made by the original members who reunited after a long separation.
Another band that did not have its albums organized in a chronological order is the Bangles, where the last album Sweetheart of the Sun released in 2012 was placed after the album Different Light. We are not sufficiently familiar with the music to explain that clear violation of the chronological order, and neither is the nine-year-old child who inspired us to analyze that band.
In the case of U2 we failed to find any reasonable explanation the exclusion of the album War from the rest of the band's albums, but it also shows that the music did not change drastically during the late '80s and early '90s.
Jimi Hendrix's albums sound to the computer like this:
For a more modern example, we analyzed Taylor Swift's albums.
And also Eminem.
That is what the computer thinks of Bob Seger's albums:
We didn't have all of Elvis' albums, but from those we did have, that's what the computer came up with:
