SpatialNet with Binaural Loss Function for Correcting Binaural Signal Matching Outputs under Head Rotations

TL;DR

This paper introduces a deep learning framework using SpatialNet and a binaural loss to improve binaural signal matching accuracy during head rotations, addressing artifacts caused by traditional methods in virtual auditory environments.

Contribution

The paper presents a novel integration of deep learning with BSM-MagLS, guided by perceptual binaural loss, to enhance spatial accuracy under head rotations in binaural reproduction.

Findings

Robust performance across head rotations in simulations

Effective artifact mitigation in reverberant environments

Improved binaural signal matching accuracy

Abstract

Binaural reproduction is gaining increasing attention with the rise of devices such as virtual reality headsets, smart glasses, and head-tracked headphones. Achieving accurate binaural signals with these systems is challenging, as they often employ arbitrary microphone arrays with limited spatial resolution. The Binaural Signals Matching with Magnitude Least-Squares (BSM-MagLS) method was developed to address limitations of earlier BSM formulations, improving reproduction at high frequencies and under head rotation. However, its accuracy still degrades as head rotation increases, resulting in spatial and timbral artifacts, particularly when the virtual listener's ear moves farther from the nearest microphones. In this work, we propose the integration of deep learning with BSM-MagLS to mitigate these degradations. A post-processing framework based on the SpatialNet network is employed, leveraging its ability to process spatial information effectively and guided by both signal-level loss and a perceptually motivated binaural loss derived from a theoretical model of human binaural hearing. The effectiveness of the approach is investigated in a simulation study with a six-microphone semicircular array, showing its ability to perform robustly across head rotations. These findings are further studied in a listening experiment across different reverberant acoustic environments and head rotations, demonstrating that the proposed framework effectively mitigates BSM-MagLS degradations and provides robust correction across substantial head rotations.