Array Configuration-Agnostic Personalized Speech Enhancement using Long-Short-Term Spatial Coherence

TL;DR

This paper introduces an array configuration-agnostic multichannel personalized speech enhancement system that leverages a novel spatial feature, enabling effective speech suppression across diverse microphone array setups in challenging acoustic environments.

Contribution

The paper proposes a new spatial feature called long short term spatial coherence (LSTSC) for array-agnostic speech enhancement using convolutional recurrent networks, reducing dependency on specific array configurations.

Findings

The proposed system outperforms baselines in unseen room and array configurations.

LSTSC feature reduces computational cost while maintaining performance.

The system effectively suppresses TV noise and competing speakers in various scenarios.

Abstract

Personalized speech enhancement has been a field of active research for suppression of speechlike interferers such as competing speakers or TV dialogues. Compared with single channel approaches, multichannel PSE systems can be more effective in adverse acoustic conditions by leveraging the spatial information in microphone signals. However, the implementation of multichannel PSEs to accommodate a wide range of array topology in household applications can be challenging. To develop an array configuration agnostic PSE system, we define a spatial feature termed the long short term spatial coherence as the input feature to a convolutional recurrent network to monitor the voice activity of the target speaker. As another refinement, an equivalent rectangular bandwidth scaled LSTSC feature can be used to reduce the computational cost. Experiments were conducted to compare the proposed PSE systems, including the complete and the simplified versions with two baselines using unseen room responses and array configurations in the presence of TV noise and competing speakers. The results demonstrated that the proposed multichannel PSE network trained with the LSTSC feature achieved superior enhancement performance without precise knowledge of the array configurations and room responses.