On Time Domain Conformer Models for Monaural Speech Separation in Noisy Reverberant Acoustic Environments

TL;DR

This paper explores the application of time domain conformer models for monaural speech separation in noisy, reverberant environments, demonstrating their efficiency and effectiveness compared to existing models.

Contribution

It introduces time domain conformers with subsampling layers for speech separation, showing improved efficiency and state-of-the-art performance on benchmark datasets.

Findings

Conformers outperform dual-path networks for shorter signals.

Subsampling layers enhance computational efficiency.

Achieved 14.6 dB and 21.2 dB SISDR improvements on benchmarks.

Abstract

Speech separation remains an important topic for multi-speaker technology researchers. Convolution augmented transformers (conformers) have performed well for many speech processing tasks but have been under-researched for speech separation. Most recent state-of-the-art (SOTA) separation models have been time-domain audio separation networks (TasNets). A number of successful models have made use of dual-path (DP) networks which sequentially process local and global information. Time domain conformers (TD-Conformers) are an analogue of the DP approach in that they also process local and global context sequentially but have a different time complexity function. It is shown that for realistic shorter signal lengths, conformers are more efficient when controlling for feature dimension. Subsampling layers are proposed to further improve computational efficiency. The best TD-Conformer achieves 14.6 dB and 21.2 dB SISDR improvement on the WHAMR and WSJ0-2Mix benchmarks, respectively.