Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separation

TL;DR

This paper introduces DPTNet, a dual-path transformer model that directly models speech sequences with context-awareness, improving end-to-end monaural speech separation performance over existing methods.

Contribution

The paper proposes a novel dual-path transformer network that incorporates a recurrent component to learn sequence order without positional encodings, enhancing speech separation.

Findings

Achieves 20.6 dB SDR on WSj0-2mix dataset

Outperforms current state-of-the-art methods

Efficiently models long speech sequences

Abstract

The dominant speech separation models are based on complex recurrent or convolution neural network that model speech sequences indirectly conditioning on context, such as passing information through many intermediate states in recurrent neural network, leading to suboptimal separation performance. In this paper, we propose a dual-path transformer network (DPTNet) for end-to-end speech separation, which introduces direct context-awareness in the modeling for speech sequences. By introduces a improved transformer, elements in speech sequences can interact directly, which enables DPTNet can model for the speech sequences with direct context-awareness. The improved transformer in our approach learns the order information of the speech sequences without positional encodings by incorporating a recurrent neural network into the original transformer. In addition, the structure of dual paths makes our model efficient for extremely long speech sequence modeling. Extensive experiments on benchmark datasets show that our approach outperforms the current state-of-the-arts (20.6 dB SDR on the public WSj0-2mix data corpus).