MMDenseLSTM: An efficient combination of convolutional and recurrent neural networks for audio source separation

TL;DR

This paper introduces MMDenseLSTM, a novel neural network architecture combining CNN and LSTM for audio source separation, achieving superior accuracy and efficiency over previous models and ideal masks.

Contribution

The paper presents an innovative architecture integrating multi-scale LSTM with skip connections, improving audio source separation performance and efficiency.

Findings

Outperforms MMDenseNet, LSTM, and their blend in accuracy.

Uses fewer parameters and less processing time.

Achieves better results than ideal binary masks in singing voice separation.

Abstract

Deep neural networks have become an indispensable technique for audio source separation (ASS). It was recently reported that a variant of CNN architecture called MMDenseNet was successfully employed to solve the ASS problem of estimating source amplitudes, and state-of-the-art results were obtained for DSD100 dataset. To further enhance MMDenseNet, here we propose a novel architecture that integrates long short-term memory (LSTM) in multiple scales with skip connections to efficiently model long-term structures within an audio context. The experimental results show that the proposed method outperforms MMDenseNet, LSTM and a blend of the two networks. The number of parameters and processing time of the proposed model are significantly less than those for simple blending. Furthermore, the proposed method yields better results than those obtained using ideal binary masks for a singing voice separation task.