What does music look like?

Matching diagrams provide a way to see structure in music. Take the following picture built from the first line of Mary Had a Little Lamb. Each arc connects two matching passages, where a "match" means that they contain the same sequence of notes. To clarify the connection between the visualization and the song, I've shown the score beneath the arcs.

diagram of Mary Had a Little Lamb
Beginning of Mary Had a Little Lamb, patterns of 3 or more notes.

The next diagram visualizes Beethoven's Fur Elise. Again, matches are based on equality of pitch; where chords occur we consider only the top note. Despite this extremely limited definition of musical similarity, the resulting matching diagram reveals an intricate and beautiful structure.

diagram of Fur Elise
Beethoven, Fur Elise (Bagatelle in A Minor)

The picture shows how the piece begins and ends with the same passage, while a longer version of that passage repeats throughout at increasing intervals. You can also see a long stretch in the second half where that passage is not repeated at all and whose structure looks distinctly different, which corresponds well to what you hear when you listen to the music.

diagram of toreador
Bizet, "Toreador," Carmen

Not all pieces show as much exact repetition as Fur Elise. For instance, the "Toreador" song from Carmen, diagrammed above, looks completely different. Instead of a few long passages repeated exactly over and over again, it contains many repeated smaller phrases.

diagram of Bach minuet
Bach, Minuet in G Major

As a final example, consider Bach's Minuet in G Major. The diagram shows that the piece divides into two main parts, each made of a long passage played twice--or what a musician would call an "AABB" structure. It's not surprising to see this in a minuet, which shows that the matching diagram is picking out structures that correspond to conventional notions of musical form.

The diagram, however, provides much more detailed information than the simple "AABB" notation. For instance, you can see that the A and B passages are loosely related, as shown by the bundle of thin arcs connecting the two halves of the piece. And the fact that the two main arcs overlap shows that the end of the A passage is the same as B's beginning.

Although the diagrams on this page show some complex patterns and raise interesting questions, they're only a first step. A useful picture of a musical work would require, at the very least, a matching algorithm that could handle polyphony and modulated patterns. It would also be nice to add interactive capabilities, allowing users to change the scale of the diagrams, view selected groups of matches, and see the score underlying any pattern. Finally, it would be wonderful if the computer could animate the diagrams while playing the underlying music.

Intro | Musical structure | Patterns in encoded data | Related methods | Implementation
Martin Wattenberg, July 1999