Multi-Band Multi-Resolution Fully Convolutional Neural Networks for Singing Voice Separation

TL;DR

This paper introduces a multi-band multi-resolution convolutional neural network that processes different frequency bands of audio spectrograms with tailored resolutions, improving singing voice separation performance efficiently.

Contribution

It proposes a novel neural network architecture that processes spectrogram frequency bands with different resolutions and filter configurations, enhancing separation accuracy.

Findings

Outperforms existing deep neural networks in singing voice separation

Uses fewer parameters while achieving better results

Effectively captures low and high frequency information separately

Abstract

Deep neural networks with convolutional layers usually process the entire spectrogram of an audio signal with the same time-frequency resolutions, number of filters, and dimensionality reduction scale. According to the constant-Q transform, good features can be extracted from audio signals if the low frequency bands are processed with high frequency resolution filters and the high frequency bands with high time resolution filters. In the spectrogram of a mixture of singing voices and music signals, there is usually more information about the voice in the low frequency bands than the high frequency bands. These raise the need for processing each part of the spectrogram differently. In this paper, we propose a multi-band multi-resolution fully convolutional neural network (MBR-FCN) for singing voice separation. The MBR-FCN processes the frequency bands that have more information about the target signals with more filters and smaller dimentionality reduction scale than the bands with less information. Furthermore, the MBR-FCN processes the low frequency bands with high frequency resolution filters and the high frequency bands with high time resolution filters. Our experimental results show that the proposed MBR-FCN with very few parameters achieves better singing voice separation performance than other deep neural networks.