Generating Separated Singing Vocals Using a Diffusion Model Conditioned on Music Mixtures

TL;DR

This paper introduces a diffusion model for singing voice separation from music mixtures, demonstrating improved performance, controllable output quality, and the ability for user refinement, advancing generative approaches in source separation.

Contribution

The work presents a novel diffusion-based method for singing voice separation conditioned on music mixtures, outperforming prior generative models and enabling user-controlled output refinement.

Findings

Achieves competitive scores against non-generative baselines.

Enables user control over quality-efficiency trade-off.

Provides an ablation study on sampling algorithm parameters.

Abstract

Separating the individual elements in a musical mixture is an essential process for music analysis and practice. While this is generally addressed using neural networks optimized to mask or transform the time-frequency representation of a mixture to extract the target sources, the flexibility and generalization capabilities of generative diffusion models are giving rise to a novel class of solutions for this complicated task. In this work, we explore singing voice separation from real music recordings using a diffusion model which is trained to generate the solo vocals conditioned on the corresponding mixture. Our approach improves upon prior generative systems and achieves competitive objective scores against non-generative baselines when trained with supplementary data. The iterative nature of diffusion sampling enables the user to control the quality-efficiency trade-off, and also refine the output when needed. We present an ablation study of the sampling algorithm, highlighting the effects of the user-configurable parameters.