One model to enhance them all: array geometry agnostic multi-channel personalized speech enhancement

TL;DR

This paper introduces a novel array-geometry-agnostic multi-channel personalized speech enhancement model that leverages spatial information to improve speech quality and recognition accuracy across various microphone array configurations.

Contribution

The paper proposes a new causal multi-channel PSE model that is independent of microphone array geometry, enhancing performance and generalization to unseen array configurations.

Findings

Outperforms geometry-specific models in speech quality

Improves automatic speech recognition accuracy

Effective on unseen microphone array geometries

Abstract

With the recent surge of video conferencing tools usage, providing high-quality speech signals and accurate captions have become essential to conduct day-to-day business or connect with friends and families. Single-channel personalized speech enhancement (PSE) methods show promising results compared with the unconditional speech enhancement (SE) methods in these scenarios due to their ability to remove interfering speech in addition to the environmental noise. In this work, we leverage spatial information afforded by microphone arrays to improve such systems' performance further. We investigate the relative importance of speaker embeddings and spatial features. Moreover, we propose a new causal array-geometry-agnostic multi-channel PSE model, which can generate a high-quality enhanced signal from arbitrary microphone geometry. Experimental results show that the proposed geometry agnostic model outperforms the model trained on a specific microphone array geometry in both speech quality and automatic speech recognition accuracy. We also demonstrate the effectiveness of the proposed approach for unseen array geometries.