CrossSong is a music-based puzzle game in which the goal is to recognize the component parts of mash-ups. The player is faced with a grid of tiles, each representing a mash-up between two songs, and they must rearrange the tiles so that the tiles in each row and each column contain parts of the same song. The project was mostly about making a fun game, but research contributions also include an algorithm for finding an optimal combination of mashups. Project page includes a link to the playable game!
My PhD thesis considered listener disagreements in the analysis of musical structure, and looked at the issue from a number of viewpoints in different disciplines: music information retrieval, music theory and music perception and cognition. The following four projects comprised the main chapters:
We conducted an online listening study to test how the focus of a listener could affect their perception of structure. We found that by manipulating someone’s attention, whether overtly or obliquely, we could influence the salience of boundaries and the structural groupings they prefered.
In this project, we proposed a method for estimating what musical attributes someone was paying attention to when they analyzed a piece of music. In other words, we sought to find a correlation between a recording and a listener’s annotation of it. We used a simple quadratic programming algorithm and obtained some interesting section-by-section maps of pieces.
How well does acoustic novelty account for boundary indications in an annotated corpus? We looked at how peaks in novelty (at various timescales and in various musical features) correlated with boundary indications. We found that novelty is a necessary but not sufficient condition for being a boundary.
How can two people listen to the same music, but disagree about its structure? In this case study co-authored by Isaac Schankler, we tried to trace the evolution of analytical disagreements to understand their origin. We analyzed the same pieces of music, and then thoroughly compared our justifications for our analyses when they disagreed. The disagreements seemed to boil down to differences in attention, prior knowledge and expectation. We published the article in Music Theory Online, which meant we could include all the relevant audio files, videos and illustrations.
After several years of running the evaluations of structural analysis at MIREX, what can we learn about which evaluation metrics are useful, which are redundant, and which songs are hardest to analyze? This ISMIR paper focused on these questions. The work was published as “open research”, meaning that all the tools and data used to produce the article are provided in a public repository.
For my Master’s thesis, submitted in August 2010, I conducted a comparative evaluation of a handful of algorithms that produce formal analyses of music on a diverse set of corpora, including a new corpus of public domain music.
The Structural Analysis of Large Amounts of Music Information (SALAMI) project is a multi-national effort to produce a corpus of analyses of hundreds of thousands of pieces of music. I oversaw the first phase of this project: the creation of a huge ground truth dataset. As part of my work, I developed and tested a novel annotation format, evaluated and hired annotators, oversaw months of data collection and presented the data at ISMIR 2011. I continue to manage and develop the dataset, which was released to the public in February 2012. It’s available now on GitHub.