AMDM-SE: Attention-based Multichannel Diffusion Model for Speech Enhancement

TL;DR

This paper introduces AMDM-SE, a novel attention-based multichannel diffusion model that significantly improves speech enhancement by effectively utilizing spatial information from multiple microphones.

Contribution

It proposes a new cross-channel time-frequency attention mechanism within a diffusion framework for multichannel speech enhancement, addressing a key gap in current research.

Findings

Outperforms single-channel and non-attention multichannel baselines

Demonstrates the effectiveness of multichannel attention in noise reduction

Shows significant improvements on the CHiME-3 benchmark

Abstract

Diffusion models have recently achieved impressive results in reconstructing images from noisy inputs, and similar ideas have been applied to speech enhancement by treating time-frequency representations as images. With the ubiquity of multi-microphone devices, we extend state-of-the-art diffusion-based methods to exploit multichannel inputs for improved performance. Multichannel diffusion-based enhancement remains in its infancy, with prior work making limited use of advanced mechanisms such as attention for spatial modeling - a gap addressed in this paper. We propose AMDM-SE, an Attention-based Multichannel Diffusion Model for Speech Enhancement, designed specifically for noise reduction. AMDM-SE leverages spatial inter-channel information through a novel cross-channel time-frequency attention block, enabling faithful reconstruction of fine-grained signal details within a generative diffusion framework. On the CHiME-3 benchmark, AMDM-SE outperforms both a single-channel diffusion baseline and a multichannel model without attention, as well as a strong DNN-based predictive method. Simulated-data experiments further underscore the importance of the proposed multichannel attention mechanism. Overall, our results show that incorporating targeted multichannel attention into diffusion models substantially improves noise reduction. While multichannel diffusion-based speech enhancement is still an emerging field, our work contributes a new and complementary approach to the growing body of research in this direction.