Investigating U-Nets with various Intermediate Blocks for Spectrogram-based Singing Voice Separation

TL;DR

This paper evaluates various intermediate blocks in U-Net architectures for singing voice separation, demonstrating that certain configurations significantly improve SDR performance on the MUSDB dataset.

Contribution

It introduces and compares multiple intermediate spectrogram transformation blocks within U-Nets for SVS, achieving state-of-the-art results with a specific block type.

Findings

A particular convolutional and fully-connected block achieves 0.9 dB SDR improvement.

U-Net variants with different blocks are systematically compared.

The best model sets a new SDR benchmark on MUSDB for SVS.

Abstract

Singing Voice Separation (SVS) tries to separate singing voice from a given mixed musical signal. Recently, many U-Net-based models have been proposed for the SVS task, but there were no existing works that evaluate and compare various types of intermediate blocks that can be used in the U-Net architecture. In this paper, we introduce a variety of intermediate spectrogram transformation blocks. We implement U-nets based on these blocks and train them on complex-valued spectrograms to consider both magnitude and phase. These networks are then compared on the SDR metric. When using a particular block composed of convolutional and fully-connected layers, it achieves state-of-the-art SDR on the MUSDB singing voice separation task by a large margin of 0.9 dB. Our code and models are available online.