Investigating the efficacy of music version retrieval systems for setlist identification

TL;DR

This paper evaluates music version retrieval systems for setlist identification in live performances, introduces a new dataset, and analyzes their effectiveness across genres and database sizes.

Contribution

It proposes an end-to-end workflow for setlist identification, compares three version identification systems, and provides a new annotated dataset for evaluation.

Findings

Identifies 68% of segments on average, with performance varying by genre.

Expanding the reference database to 56.8k songs still achieves 56% identification.

The approach is effective across different audio qualities and genres.

Abstract

The setlist identification (SLI) task addresses a music recognition use case where the goal is to retrieve the metadata and timestamps for all the tracks played in live music events. Due to various musical and non-musical changes in live performances, developing automatic SLI systems is still a challenging task that, despite its industrial relevance, has been under-explored in the academic literature. In this paper, we propose an end-to-end workflow that identifies relevant metadata and timestamps of live music performances using a version identification system. We compare 3 of such systems to investigate their suitability for this particular task. For developing and evaluating SLI systems, we also contribute a new dataset that contains 99.5h of concerts with annotated metadata and timestamps, along with the corresponding reference set. The dataset is categorized by audio qualities and genres to analyze the performance of SLI systems in different use cases. Our approach can identify 68% of the annotated segments, with values ranging from 35% to 77% based on the genre. Finally, we evaluate our approach against a database of 56.8k songs to illustrate the effect of expanding the reference set, where we can still identify 56% of the annotated segments.