Sound propagation in realistic interactive 3D scenes with parameterized sources using deep neural operators

TL;DR

This paper introduces a deep neural operator approach for real-time sound propagation simulation in complex 3D scenes with moving sources, significantly improving efficiency and accuracy over traditional methods.

Contribution

It presents a novel deep operator network model that rapidly predicts wave fields in realistic 3D environments, enabling real-time sound simulation with moving sources.

Findings

Achieved millisecond-scale sound propagation predictions.

Maintained low root mean squared errors (0.02-0.10 Pa) compared to reference solutions.

Demonstrated effectiveness across complex scene geometries.

Abstract

We address the challenge of sound propagation simulations in 3D virtual rooms with moving sources, which have applications in virtual/augmented reality, game audio, and spatial computing. Solutions to the wave equation can describe wave phenomena such as diffraction and interference. However, simulating them using conventional numerical discretization methods with hundreds of source and receiver positions is intractable, making stimulating a sound field with moving sources impractical. To overcome this limitation, we propose using deep operator networks to approximate linear wave-equation operators. This enables the rapid prediction of sound propagation in realistic 3D acoustic scenes with moving sources, achieving millisecond-scale computations. By learning a compact surrogate model, we avoid the offline calculation and storage of impulse responses for all relevant source/listener pairs. Our experiments, including various complex scene geometries, show good agreement with reference solutions, with root mean squared errors ranging from 0.02 Pa to 0.10 Pa. Notably, our method signifies a paradigm shift as no prior machine learning approach has achieved precise predictions of complete wave fields within realistic domains. We anticipate that our findings will drive further exploration of deep neural operator methods, advancing research in immersive user experiences within virtual environments.$