Attention-Based Beamformer For Multi-Channel Speech Enhancement

TL;DR

This paper introduces an attention-based approach to improve multi-channel speech enhancement by dynamically estimating spatial covariance matrices, effectively handling moving sources and outperforming traditional methods.

Contribution

It proposes a novel attention-based mechanism for SCM estimation in MVDR beamforming, incorporating spatial information with inplace convolution and frequency-independent LSTM, optimized end-to-end.

Findings

Outperforms baseline methods in speech enhancement quality.

Reduces computational complexity and model parameters.

Effectively handles moving sources in multi-channel scenarios.

Abstract

Minimum Variance Distortionless Response (MVDR) is a classical adaptive beamformer that theoretically ensures the distortionless transmission of signals in the target direction, which makes it popular in real applications. Its noise reduction performance actually depends on the accuracy of the noise and speech spatial covariance matrices (SCMs) estimation. Time-frequency masks are often used to compute these SCMs. However, most mask-based beamforming methods typically assume that the sources are stationary, ignoring the case of moving sources, which leads to performance degradation. In this paper, we propose an attention-based mechanism to calculate the speech and noise SCMs and then apply MVDR to obtain the enhanced speech. To fully incorporate spatial information, the inplace convolution operator and frequency-independent LSTM are applied to facilitate SCMs estimation. The model is optimized in an end-to-end manner. Experiments demonstrate that the proposed method outperforms baselines with reduced computation and fewer parameters under various conditions.