NeuralSound: Learning-based Modal Sound Synthesis With Acoustic Transfer

TL;DR

NeuralSound introduces a fast, learning-based modal sound synthesis method combining a mixed vibration solver and an acoustic transfer network, achieving high-quality sound generation in under a second.

Contribution

The paper presents a novel integrated approach for modal sound synthesis that combines a mixed vibration solver with an end-to-end acoustic transfer network, enabling rapid and accurate sound generation.

Findings

Sound synthesis time is less than one second on a GTX 3080 Ti GPU.

High sound quality close to ground truth numerical methods.

Effective across different objects and materials.

Abstract

We present a novel learning-based modal sound synthesis approach that includes a mixed vibration solver for modal analysis and an end-to-end sound radiation network for acoustic transfer. Our mixed vibration solver consists of a 3D sparse convolution network and a Locally Optimal Block Preconditioned Conjugate Gradient module (LOBPCG) for iterative optimization. Moreover, we highlight the correlation between a standard modal vibration solver and our network architecture. Our radiation network predicts the Far-Field Acoustic Transfer maps (FFAT Maps) from the surface vibration of the object. The overall running time of our learning method for any new object is less than one second on a GTX 3080 Ti GPU while maintaining a high sound quality close to the ground truth that is computed using standard numerical methods. We also evaluate the numerical accuracy and perceptual accuracy of our sound synthesis approach on different objects corresponding to various materials.