Spectrogram-channels u-net: a source separation model viewing each channel as the spectrogram of each source

TL;DR

This paper introduces Spectrogram-Channels U-Net, a novel source separation model that treats each output channel as a spectrogram of a separated source, achieving state-of-the-art results in singing voice and multi-instrument separation.

Contribution

It presents a new spectrogram-based U-Net model with a volume-balancing loss function, adaptable to various source separation tasks.

Findings

Achieved state-of-the-art separation performance

Effective for both singing voice and multi-instrument separation

Introduced a volume-balancing loss function

Abstract

Sound source separation has attracted attention from Music Information Retrieval(MIR) researchers, since it is related to many MIR tasks such as automatic lyric transcription, singer identification, and voice conversion. In this paper, we propose an intuitive spectrogram-based model for source separation by adapting U-Net. We call it Spectrogram-Channels U-Net, which means each channel of the output corresponds to the spectrogram of separated source itself. The proposed model can be used for not only singing voice separation but also multi-instrument separation by changing only the number of output channels. In addition, we propose a loss function that balances volumes between different sources. Finally, we yield performance that is state-of-the-art on both separation tasks.