Deep learning based spatial aliasing reduction in beamforming for audio capture

TL;DR

This paper introduces a deep learning method using a U-Net architecture to reduce spatial aliasing in beamforming, significantly improving audio capture quality in multi-microphone systems.

Contribution

It presents a novel deep learning approach for spatial aliasing mitigation in beamforming, employing a U-Net to predict de-aliasing filters for enhanced spatial audio capture.

Findings

Significant objective and perceptual improvements over conventional beamforming.

Effective reduction of spatial aliasing in stereo and Ambisonics scenarios.

Demonstrates deep learning's potential in multi-microphone spatial audio enhancement.

Abstract

Spatial aliasing affects spaced microphone arrays, causing directional ambiguity above certain frequencies, degrading spatial and spectral accuracy of beamformers. Given the limitations of conventional signal processing and the scarcity of deep learning approaches to spatial aliasing mitigation, we propose a novel approach using a U-Net architecture to predict a signal-dependent de-aliasing filter, which reduces aliasing in conventional beamforming for spatial capture. Two types of multichannel filters are considered, one which treats the channels independently and a second one that models cross-channel dependencies. The proposed approach is evaluated in two common spatial capture scenarios: stereo and first-order Ambisonics. The results indicate a very significant improvement, both objective and perceptual, with respect to conventional beamforming. This work shows the potential of deep learning to reduce aliasing in beamforming, leading to improvements in multi-microphone setups.