Long-Context Modeling Networks for Monaural Speech Enhancement: A Comparative Study

TL;DR

This paper compares various long-context neural network architectures, including Transformer, Conformer, Mamba, and xLSTM, for monaural speech enhancement, highlighting their performance and efficiency differences.

Contribution

It provides the first comprehensive comparison of these backbones within a unified speech enhancement framework, including the novel application of xLSTM.

Findings

Mamba outperforms others in efficiency and performance.

xLSTM shows promising results but is slower.

Mamba is particularly effective for long speech inputs.

Abstract

Advanced long-context modeling backbone networks, such as Transformer, Conformer, and Mamba, have demonstrated state-of-the-art performance in speech enhancement. However, a systematic and comprehensive comparative study of these backbones within a unified speech enhancement framework remains lacking. In addition, xLSTM, a more recent and efficient variant of LSTM, has shown promising results in language modeling and as a general-purpose vision backbone. In this paper, we investigate the capability of xLSTM in speech enhancement, and conduct a comprehensive comparison and analysis of the Transformer, Conformer, Mamba, and xLSTM backbones within a unified framework, considering both causal and noncausal configurations. Overall, xLSTM and Mamba achieve better performance than Transformer and Conformer. Mamba demonstrates significantly superior training and inference efficiency, particularly for long speech inputs, whereas xLSTM suffers from the slowest processing speed.