Neural Ambisonics encoding for compact irregular microphone arrays

TL;DR

This paper introduces a deep neural network-based Ambisonics encoding method for irregular microphone arrays, enabling improved spatial audio capture in virtual reality and telepresence applications.

Contribution

It presents a novel DNN approach with a U-Net structure for Ambisonics encoding on irregular arrays, outperforming traditional signal-independent encoders.

Findings

The method achieves comparable or better error metrics than conventional encoders.

Validated on irregular and regular microphone arrays in simulated reverberant scenes.

Demonstrates potential for flexible spatial audio recording with irregular microphone configurations.

Abstract

Ambisonics encoding of microphone array signals can enable various spatial audio applications, such as virtual reality or telepresence, but it is typically designed for uniformly-spaced spherical microphone arrays. This paper proposes a method for Ambisonics encoding that uses a deep neural network (DNN) to estimate a signal transform from microphone inputs to Ambisonics signals. The approach uses a DNN consisting of a U-Net structure with a learnable preprocessing as well as a loss function consisting of mean average error, spatial correlation, and energy preservation components. The method is validated on two microphone arrays with regular and irregular shapes having four microphones, on simulated reverberant scenes with multiple sources. The results of the validation show that the proposed method can meet or exceed the performance of a conventional signal-independent Ambisonics encoder on a number of error metrics.