Resounding Acoustic Fields with Reciprocity

TL;DR

This paper introduces Versa, a physics-inspired, reciprocity-based method for estimating room impulse responses at arbitrary emitter locations, significantly enhancing immersive spatial sound experiences in virtual environments.

Contribution

The paper presents a novel reciprocity-based approach called Versa for acoustic field learning, addressing challenges in dynamic sound modeling and enabling dense virtual emitter sampling.

Findings

Versa improves acoustic field estimation accuracy on simulated datasets.

Versa enhances perceptual spatial sound quality in user studies.

The method effectively generalizes to real-world environments.

Abstract

Achieving immersive auditory experiences in virtual environments requires flexible sound modeling that supports dynamic source positions. In this paper, we introduce a task called resounding, which aims to estimate room impulse responses at arbitrary emitter location from a sparse set of measured emitter positions, analogous to the relighting problem in vision. We leverage the reciprocity property and introduce Versa, a physics-inspired approach to facilitating acoustic field learning. Our method creates physically valid samples with dense virtual emitter positions by exchanging emitter and listener poses. We also identify challenges in deploying reciprocity due to emitter/listener gain patterns and propose a self-supervised learning approach to address them. Results show that Versa substantially improve the performance of acoustic field learning on both simulated and real-world datasets across different metrics. Perceptual user studies show that Versa can greatly improve the immersive spatial sound experience. Code, dataset and demo videos are available on the project website: https://waves.seas.upenn.edu/projects/versa.