Multi-Scale Feature Fusion Transformer Network for End-to-End Single Channel Speech Separation

TL;DR

This paper introduces MSFFT-Net, a multi-scale feature fusion transformer for single-channel speech separation, outperforming previous dual-path models by capturing local and global context through parallel processing paths.

Contribution

The paper proposes a novel multi-scale feature fusion transformer network with parallel paths and iterative intra- and inter-chunk operations for improved speech separation.

Findings

Achieved state-of-the-art SI-SNRi scores on WSJ0-2mix dataset.

Outperformed original dual-path models without data augmentation.

Demonstrated better results with multiple parallel processing paths.

Abstract

Recently studies on time-domain audio separation networks (TasNets) have made a great stride in speech separation. One of the most representative TasNets is a network with a dual-path segmentation approach. However, the original model called DPRNN used a fixed feature dimension and unchanged segment size throughout all layers of the network. In this paper, we propose a multi-scale feature fusion transformer network (MSFFT-Net) based on the conventional dual-path structure for single-channel speech separation. Unlike the conventional dual-path structure where only one processing path exists, adopting several iterative blocks with alternative intra-chunk and inter-chunk operations to capture local and global context information, the proposed MSFFT-Net has multiple parallel processing paths where the feature information can be exchanged between multiple parallel processing paths. Experiments show that our proposed networks based on multi-scale feature fusion structure have achieved better results than the original dual-path model on the benchmark dataset-WSJ0-2mix, where the SI-SNRi score of MSFFT-3P is 20.7dB (1.47% improvement), and MSFFT-2P is 21.0dB (3.45% improvement), which achieves SOTA on WSJ0-2mix without any data augmentation method.