Solos: A Dataset for Audio-Visual Music Analysis

TL;DR

The paper introduces Solos, a high-quality, instrument-diverse audio-visual dataset from YouTube performances, designed to advance machine learning tasks like source separation, localization, and cross-modal analysis.

Contribution

It provides a clean, manually verified dataset covering all instruments in URMP, enabling improved training and evaluation of audio-visual machine learning models.

Findings

Models trained on Solos perform well on URMP source separation tasks.

Solos dataset is cleaner and more comprehensive than previous datasets.

The dataset supports multiple audio-visual analysis tasks.

Abstract

In this paper, we present a new dataset of music performance videos which can be used for training machine learning methods for multiple tasks such as audio-visual blind source separation and localization, cross-modal correspondences, cross-modal generation and, in general, any audio-visual self-supervised task. These videos, gathered from YouTube, consist of solo musical performances of 13 different instruments. Compared to previously proposed audio-visual datasets, Solos is cleaner since a big amount of its recordings are auditions and manually checked recordings, ensuring there is no background noise nor effects added in the video post-processing. Besides, it is, up to the best of our knowledge, the only dataset that contains the whole set of instruments present in the URMP\cite{URPM} dataset, a high-quality dataset of 44 audio-visual recordings of multi-instrument classical music pieces with individual audio tracks. URMP was intented to be used for source separation, thus, we evaluate the performance on the URMP dataset of two different source-separation models trained on Solos. The dataset is publicly available at https://juanfmontesinos.github.io/Solos/