Resonance4D: Frequency-Domain Motion Supervision for Preset-Free Physical Parameter Learning in 4D Dynamic Physical Scene Simulation

TL;DR

Resonance4D introduces a frequency-domain, physics-driven 4D simulation framework that efficiently enforces dynamic consistency without dense temporal generation, enabling realistic scene simulation with reduced computational cost.

Contribution

It proposes Dual-domain Motion Supervision combining spatial and frequency-domain constraints, and a method for automatic object-part decomposition for full-parameter physical recovery.

Findings

Achieves high physical fidelity and motion consistency in synthetic and real scenes.

Reduces GPU memory usage from over 35GB to around 20GB, enabling simulation on consumer-grade GPUs.

Enables stable full-parameter physical recovery with automatic object-part segmentation.

Abstract

Physics-driven 4D dynamic simulation from static 3D scenes remains constrained by an overlooked contradiction: reliable motion supervision often relies on online video diffusion or optical-flow pipelines whose computational cost exceeds that of the simulator itself. Existing methods further simplify inverse physical modeling by optimizing only partial material parameters, limiting realism in scenes with complex materials and dynamics. We present Resonance4D, a physics-driven 4D dynamic simulation framework that couples 3D Gaussian Splatting with the Material Point Method through lightweight yet physically expressive supervision. Our key insight is that dynamic consistency can be enforced without dense temporal generation by jointly constraining motion in complementary domains. To this end, we introduce Dual-domain Motion Supervision (DMS), which combines spatial structural consistency for local deformation with frequency-domain spectral consistency for oscillatory and global dynamic patterns, substantially reducing training cost and memory overhead while preserving physically meaningful motion cues. To enable stable full-parameter physical recovery, we further combine zero-shot text-prompted segmentation with simulation-guided initialization to automatically decompose Gaussians into object-part-level regions and support joint optimization of full material parameters. Experiments on both synthetic and real scenes show that Resonance4D achieves strong physical fidelity and motion consistency while reducing peak GPU memory from over 35\,GB to around 20\,GB, enabling high-fidelity physics-driven 4D simulation on a single consumer-grade GPU.