Retrieval-Augmented Neural Field for HRTF Upsampling and Personalization

TL;DR

This paper introduces RANF, a retrieval-augmented neural field method that improves HRTF upsampling and personalization by leveraging similar subjects' data, achieving notable results with limited measurements.

Contribution

The paper presents a novel retrieval-augmented neural field approach that enhances HRTF upsampling and personalization from minimal measurements, outperforming existing methods.

Findings

RANF improves HRTF upsampling accuracy.

RANF outperforms baseline methods on SONICOM dataset.

RANF contributed to winning the listener acoustic personalization challenge 2024.

Abstract

Head-related transfer functions (HRTFs) with dense spatial grids are desired for immersive binaural audio generation, but their recording is time-consuming. Although HRTF spatial upsampling has shown remarkable progress with neural fields, spatial upsampling only from a few measured directions, e.g., 3 or 5 measurements, is still challenging. To tackle this problem, we propose a retrieval-augmented neural field (RANF). RANF retrieves a subject whose HRTFs are close to those of the target subject from a dataset. The HRTF of the retrieved subject at the desired direction is fed into the neural field in addition to the sound source direction itself. Furthermore, we present a neural network that can efficiently handle multiple retrieved subjects, inspired by a multi-channel processing technique called transform-average-concatenate. Our experiments confirm the benefits of RANF on the SONICOM dataset, and it is a key component in the winning solution of Task 2 of the listener acoustic personalization challenge 2024.