Refining music sample identification with a self-supervised graph neural network

TL;DR

This paper introduces a lightweight graph neural network-based system for music sample identification that is robust to common audio modifications, achieving high accuracy with fewer parameters and a two-stage retrieval process.

Contribution

It presents a novel, scalable GNN architecture with contrastive learning for ASID, and a two-stage retrieval method including a cross-attention classifier, along with new benchmarks for short queries.

Findings

Achieves 44.2% mAP with only 9% of parameters of state-of-the-art

Introduces a two-stage retrieval process improving accuracy

Provides new annotated dataset for short query evaluation

Abstract

Automatic sample identification (ASID), the detection and identification of portions of audio recordings that have been reused in new musical works, is an essential but challenging task in the field of audio query-based retrieval. While a related task, audio fingerprinting, has made significant progress in accurately retrieving musical content under "real world" (noisy, reverberant) conditions, ASID systems struggle to identify samples that have undergone musical modifications. Thus, a system robust to common music production transformations such as time-stretching, pitch-shifting, effects processing, and underlying or overlaying music is an important open challenge. In this work, we propose a lightweight and scalable encoding architecture employing a Graph Neural Network within a contrastive learning framework. Our model uses only 9% of the trainable parameters compared to the current state-of-the-art system while achieving comparable performance, reaching a mean average precision (mAP) of 44.2%. To enhance retrieval quality, we introduce a two-stage approach consisting of an initial coarse similarity search for candidate selection, followed by a cross-attention classifier that rejects irrelevant matches and refines the ranking of retrieved candidates - an essential capability absent in prior models. In addition, because queries in real-world applications are often short in duration, we benchmark our system for short queries using new fine-grained annotations for the Sample100 dataset, which we publish as part of this work.