Real-time speech enhancement with dynamic attention span

TL;DR

This paper introduces a dynamic attention span mechanism in neural networks for real-time speech enhancement, allowing adaptive processing of time-variant audio signals to improve noise suppression, dereverberation, and echo cancellation.

Contribution

It proposes a novel adaptive attention span mechanism within an encoder-decoder framework for real-time speech enhancement, enhancing the model's ability to handle time-variant audio characteristics.

Findings

Improved speech quality and noise suppression in experiments.

Better tracking of time-variant factors with dynamic attention.

Enhanced performance over fixed attention models.

Abstract

For real-time speech enhancement (SE) including noise suppression, dereverberation and acoustic echo cancellation, the time-variance of the audio signals becomes a severe challenge. The causality and memory usage limit that only the historical information can be used for the system to capture the time-variant characteristics. We propose to adaptively change the receptive field according to the input signal in deep neural network based SE model. Specifically, in an encoder-decoder framework, a dynamic attention span mechanism is introduced to all the attention modules for controlling the size of historical content used for processing the current frame. Experimental results verify that this dynamic mechanism can better track time-variant factors and capture speech-related characteristics, benefiting to both interference removing and speech quality retaining.