Spatial Upsampling of Head-Related Transfer Functions Using a Physics-Informed Neural Network

TL;DR

This paper introduces a physics-informed neural network (PINN) approach for upsampling sparse head-related transfer functions (HRTFs), leveraging the Helmholtz equation to produce physically valid and generalizable HRTF estimations for personalized virtual acoustics.

Contribution

The novel PINN method integrates the Helmholtz equation into neural network training for HRTF upsampling, improving physical validity and generalization over existing data-driven approaches.

Findings

PINN outperforms SH and HRTF field methods in interpolation.

PINN generalizes well to unseen HRTFs.

Physically regularized upsampling enhances virtual acoustic realism.

Abstract

Head-related transfer function (HRTF) capture the information that a person uses to localize sound sources in space, and thus is crucial for creating personalized virtual acoustic experiences. However, practical HRTF measurement systems may only measure a person's HRTFs sparsely, and this necessitates HRTF upsampling. This paper proposes a physics-informed neural network (PINN) method for HRTF upsampling. The PINN exploits the Helmholtz equation, the governing equation of acoustic wave propagation, for regularizing the upsampling process. This helps the generation of physically valid upsamplings which generalize beyond the measured HRTF. Furthermore, the size (width and depth) of the PINN is set according to the Helmholtz equation and its solutions, the spherical harmonics (SHs). This makes the PINN have an appropriate level of expressive power and thus does not suffer from the over-fitting problem. Since the PINN is designed independent of any specific HRTF dataset, it offers more generalizability compared to pure data-driven methods. Numerical experiments confirm the better performance of the PINN method for HRTF upsampling in both interpolation and extrapolation scenarios in comparison with the SH method and the HRTF field method.