LiSenNet: Lightweight Sub-band and Dual-Path Modeling for Real-Time Speech Enhancement

TL;DR

LiSenNet is a lightweight, real-time speech enhancement network that uses sub-band and dual-path modeling to achieve competitive performance with significantly reduced computational complexity, suitable for low-resource edge devices.

Contribution

The paper introduces LiSenNet, a novel lightweight SE model with sub-band and dual-path modules, and a noise detector for adaptive processing, reducing model size and computation.

Findings

Achieves competitive speech enhancement performance

Uses only 37k parameters, half of state-of-the-art models

Requires 56M MACs per second, suitable for real-time edge deployment

Abstract

Speech enhancement (SE) aims to extract the clean waveform from noise-contaminated measurements to improve the speech quality and intelligibility. Although learning-based methods can perform much better than traditional counterparts, the large computational complexity and model size heavily limit the deployment on latency-sensitive and low-resource edge devices. In this work, we propose a lightweight SE network (LiSenNet) for real-time applications. We design sub-band downsampling and upsampling blocks and a dual-path recurrent module to capture band-aware features and time-frequency patterns, respectively. A noise detector is developed to detect noisy regions in order to perform SE adaptively and save computational costs. Compared to recent higher-resource-dependent baseline models, the proposed LiSenNet can achieve a competitive performance with only 37k parameters (half of the state-of-the-art model) and 56M multiply-accumulate (MAC) operations per second.