SonicGauss: Position-Aware Physical Sound Synthesis for 3D Gaussian Representations

TL;DR

SonicGauss introduces a novel framework that synthesizes realistic, position-aware impact sounds from 3D Gaussian representations by integrating geometric, material, and spatial acoustic features.

Contribution

It is the first to leverage 3D Gaussian representations for physical sound synthesis, combining diffusion models with feature extraction for position-dependent audio generation.

Findings

Produces realistic impact sounds conditioned on impact location.

Demonstrates robustness and generalization across object categories.

Effective on both datasets and real-world recordings.

Abstract

While 3D Gaussian representations (3DGS) have proven effective for modeling the geometry and appearance of objects, their potential for capturing other physical attributes-such as sound-remains largely unexplored. In this paper, we present a novel framework dubbed SonicGauss for synthesizing impact sounds from 3DGS representations by leveraging their inherent geometric and material properties. Specifically, we integrate a diffusion-based sound synthesis model with a PointTransformer-based feature extractor to infer material characteristics and spatial-acoustic correlations directly from Gaussian ellipsoids. Our approach supports spatially varying sound responses conditioned on impact locations and generalizes across a wide range of object categories. Experiments on the ObjectFolder dataset and real-world recordings demonstrate that our method produces realistic, position-aware auditory feedback. The results highlight the framework's robustness and generalization ability, offering a promising step toward bridging 3D visual representations and interactive sound synthesis. Project page: https://chunshi.wang/SonicGauss