U-Mamba-Net: A highly efficient Mamba-based U-net style network for noisy and reverberant speech separation

TL;DR

U-Mamba-Net is a lightweight, efficient U-Net style neural network utilizing Mamba for feature filtering, designed for effective speech separation in noisy and reverberant environments with low computational demands.

Contribution

The paper introduces U-Mamba-Net, a novel lightweight model combining Mamba and U-Net architectures for efficient speech separation in complex acoustic conditions.

Findings

Achieves improved speech separation performance on Libri2mix

Maintains low computational cost compared to existing models

Demonstrates effectiveness in noisy and reverberant environments

Abstract

The topic of speech separation involves separating mixed speech with multiple overlapping speakers into several streams, with each stream containing speech from only one speaker. Many highly effective models have emerged and proliferated rapidly over time. However, the size and computational load of these models have also increased accordingly. This is a disaster for the community, as researchers need more time and computational resources to reproduce and compare existing models. In this paper, we propose U-mamba-net: a lightweight Mamba-based U-style model for speech separation in complex environments. Mamba is a state space sequence model that incorporates feature selection capabilities. U-style network is a fully convolutional neural network whose symmetric contracting and expansive paths are able to learn multi-resolution features. In our work, Mamba serves as a feature filter, alternating with U-Net. We test the proposed model on Libri2mix. The results show that U-Mamba-Net achieves improved performance with quite low computational cost.