Music Source Restoration

TL;DR

This paper introduces Music Source Restoration (MSR), a new task to recover original music signals from degraded mixtures, supported by a novel dataset and baseline models, advancing real-world music source separation.

Contribution

The paper proposes MSR as a new task, creates the RawStems dataset with hierarchical unprocessed music stems, and establishes baseline methods demonstrating the task's feasibility.

Findings

MSR is feasible with current models like U-Former.

RawStems dataset contains 578 songs with hierarchical unprocessed stems.

Baseline results show potential for improving real-world music source separation.

Abstract

We introduce Music Source Restoration (MSR), a novel task addressing the gap between idealized source separation and real-world music production. Current Music Source Separation (MSS) approaches assume mixtures are simple sums of sources, ignoring signal degradations employed during music production like equalization, compression, and reverb. MSR models mixtures as degraded sums of individually degraded sources, with the goal of recovering original, undegraded signals. Due to the lack of data for MSR, we present RawStems, a dataset annotation of 578 songs with unprocessed source signals organized into 8 primary and 17 secondary instrument groups, totaling 354.13 hours. To the best of our knowledge, RawStems is the first dataset that contains unprocessed music stems with hierarchical categories. We consider spectral filtering, dynamic range compression, harmonic distortion, reverb and lossy codec as possible degradations, and establish U-Former as a baseline method, demonstrating the feasibility of MSR on our dataset. We release the RawStems dataset annotations, degradation simulation pipeline, training code and pre-trained models to be publicly available.