Exploring the Potential of Data-Driven Spatial Audio Enhancement Using a Single-Channel Model

TL;DR

This paper investigates whether single-channel speech enhancement models can be effectively adapted for multi-channel scenarios by processing each channel independently, aiming to simplify system design and resource requirements.

Contribution

It experimentally compares single-channel and multi-channel enhancement models, demonstrating the viability of single-channel methods for multi-channel applications with some trade-offs.

Findings

Single-channel models can be adapted for multi-channel enhancement.

Multi-channel models better preserve spatial information.

Trade-off exists between spatial preservation and intelligibility gains.

Abstract

One key aspect differentiating data-driven single- and multi-channel speech enhancement and dereverberation methods is that both the problem formulation and complexity of the solutions are considerably more challenging in the latter case. Additionally, with limited computational resources, it is cumbersome to train models that require the management of larger datasets or those with more complex designs. In this scenario, an unverified hypothesis that single-channel methods can be adapted to multi-channel scenarios simply by processing each channel independently holds significant implications, boosting compatibility between sound scene capture and system input-output formats, while also allowing modern research to focus on other challenging aspects, such as full-bandwidth audio enhancement, competitive noise suppression, and unsupervised learning. This study verifies this hypothesis by comparing the enhancement promoted by a basic single-channel speech enhancement and dereverberation model with two other multi-channel models tailored to separate clean speech from noisy 3D mixes. A direction of arrival estimation model was used to objectively evaluate its capacity to preserve spatial information by comparing the output signals with ground-truth coordinate values. Consequently, a trade-off arises between preserving spatial information with a more straightforward single-channel solution at the cost of obtaining lower gains in intelligibility scores.