Dense-TSNet: Dense Connected Two-Stage Structure for Ultra-Lightweight Speech Enhancement

TL;DR

Dense-TSNet is a novel ultra-lightweight speech enhancement model that employs a dense two-stage architecture and multi-view gaze blocks to improve performance while maintaining a small model size suitable for edge devices.

Contribution

The paper introduces Dense-TSNet, a lightweight architecture with a dense two-stage design and multi-view gaze blocks, enhancing feature refinement and performance in resource-limited settings.

Findings

Achieves effective speech enhancement with only 14K parameters.

Improves feature extraction through Multi-View Gaze Block (MVGB).

Addresses early convergence issues of baseline models.

Abstract

Speech enhancement aims to improve speech quality and intelligibility in noisy environments. Recent advancements have concentrated on deep neural networks, particularly employing the Two-Stage (TS) architecture to enhance feature extraction. However, the complexity and size of these models remain significant, which limits their applicability in resource-constrained scenarios. Designing models suitable for edge devices presents its own set of challenges. Narrow lightweight models often encounter performance bottlenecks due to uneven loss landscapes. Additionally, advanced operators such as Transformers or Mamba may lack the practical adaptability and efficiency that convolutional neural networks (CNNs) offer in real-world deployments. To address these challenges, we propose Dense-TSNet, an innovative ultra-lightweight speech enhancement network. Our approach employs a novel Dense Two-Stage (Dense-TS) architecture, which, compared to the classic Two-Stage architecture, ensures more robust refinement of the objective function in the later training stages. This leads to improved final performance, addressing the early convergence limitations of the baseline model. We also introduce the Multi-View Gaze Block (MVGB), which enhances feature extraction by incorporating global, channel, and local perspectives through convolutional neural networks (CNNs). Furthermore, we discuss how the choice of loss function impacts perceptual quality. Dense-TSNet demonstrates promising performance with a compact model size of around 14K parameters, making it particularly well-suited for deployment in resource-constrained environments.