Neural Hamiltonian Deformation Fields for Dynamic Scene Rendering

TL;DR

This paper introduces NeHaD, a physics-informed neural deformation framework based on Hamiltonian mechanics, enabling realistic and efficient dynamic scene rendering with adaptive streaming capabilities.

Contribution

NeHaD is the first method to incorporate Hamiltonian mechanics into neural Gaussian deformation fields for physically plausible dynamic scene rendering.

Findings

Achieves realistic dynamic scene rendering with physics-based deformation.

Demonstrates improved physical plausibility over prior methods.

Supports adaptive streaming with scale-aware mipmapping.

Abstract

Representing and rendering dynamic scenes with complex motions remains challenging in computer vision and graphics. Recent dynamic view synthesis methods achieve high-quality rendering but often produce physically implausible motions. We introduce NeHaD, a neural deformation field for dynamic Gaussian Splatting governed by Hamiltonian mechanics. Our key observation is that existing methods using MLPs to predict deformation fields introduce inevitable biases, resulting in unnatural dynamics. By incorporating physics priors, we achieve robust and realistic dynamic scene rendering. Hamiltonian mechanics provides an ideal framework for modeling Gaussian deformation fields due to their shared phase-space structure, where primitives evolve along energy-conserving trajectories. We employ Hamiltonian neural networks to implicitly learn underlying physical laws governing deformation. Meanwhile, we introduce Boltzmann equilibrium decomposition, an energy-aware mechanism that adaptively separates static and dynamic Gaussians based on their spatial-temporal energy states for flexible rendering. To handle real-world dissipation, we employ second-order symplectic integration and local rigidity regularization as physics-informed constraints for robust dynamics modeling. Additionally, we extend NeHaD to adaptive streaming through scale-aware mipmapping and progressive optimization. Extensive experiments demonstrate that NeHaD achieves physically plausible results with a rendering quality-efficiency trade-off. To our knowledge, this is the first exploration leveraging Hamiltonian mechanics for neural Gaussian deformation, enabling physically realistic dynamic scene rendering with streaming capabilities.