Speech enhancement deep-learning architecture for efficient edge processing

TL;DR

This paper introduces an efficient deep learning architecture for speech enhancement designed for low-power edge devices, combining multi-scale features, squeeze-excitation blocks, and a metric GAN to improve speech quality while reducing computational load.

Contribution

The proposed WSR-MGAN architecture effectively balances speech enhancement performance with computational efficiency suitable for edge devices, integrating novel multi-scale and attention mechanisms.

Findings

Outperforms baseline models on VoiceBank+DEMAND dataset

Achieves state-of-the-art results in time-domain speech enhancement

Maintains high speech quality with reduced computational complexity

Abstract

Deep learning has become a de facto method of choice for speech enhancement tasks with significant improvements in speech quality. However, real-time processing with reduced size and computations for low-power edge devices drastically degrades speech quality. Recently, transformer-based architectures have greatly reduced the memory requirements and provided ways to improve the model performance through local and global contexts. However, the transformer operations remain computationally heavy. In this work, we introduce WaveUNet squeeze-excitation Res2 (WSR)-based metric generative adversarial network (WSR-MGAN) architecture that can be efficiently implemented on low-power edge devices for noise suppression tasks while maintaining speech quality. We utilize multi-scale features using Res2Net blocks that can be related to spectral content used in speech-processing tasks. In the generator, we integrate squeeze-excitation blocks (SEB) with multi-scale features for maintaining local and global contexts along with gated recurrent units (GRUs). The proposed approach is optimized through a combined loss function calculated over raw waveform, multi-resolution magnitude spectrogram, and objective metrics using a metric discriminator. Experimental results in terms of various objective metrics on VoiceBank+DEMAND and DNS-2020 challenge datasets demonstrate that the proposed speech enhancement (SE) approach outperforms the baselines and achieves state-of-the-art (SOTA) performance in the time domain.