PhysConvex: Physics-Informed 3D Dynamic Convex Radiance Fields for Reconstruction and Simulation

TL;DR

PhysConvex introduces a physics-informed neural representation for dynamic 3D scenes that unifies visual realism with physical simulation, enabling high-fidelity reconstruction and simulation of deformable objects.

Contribution

It proposes a novel convex primitive-based representation governed by continuum mechanics, integrating physical simulation with neural radiance fields for dynamic scene reconstruction.

Findings

Outperforms existing methods in geometry and appearance reconstruction

Captures complex material deformation and boundary evolution accurately

Provides a compact, efficient simulation framework for heterogeneous materials

Abstract

Reconstructing and simulating dynamic 3D scenes with both visual realism and physical consistency remains a fundamental challenge. Existing neural representations, such as NeRFs and 3DGS, excel in appearance reconstruction but struggle to capture complex material deformation and dynamics. We propose PhysConvex, a Physics-informed 3D Dynamic Convex Radiance Field that unifies visual rendering and physical simulation. PhysConvex represents deformable radiance fields using physically grounded convex primitives governed by continuum mechanics. We introduce a boundary-driven dynamic convex representation that models deformation through vertex and surface dynamics, capturing spatially adaptive, non-uniform deformation, and evolving boundaries. To efficiently simulate complex geometries and heterogeneous materials, we further develop a reduced-order convex simulation that advects dynamic convex fields using neural skinning eigenmodes as shape- and material-aware deformation bases with time-varying reduced DOFs under Newtonian dynamics. Convex dynamics also offers compact, gap-free volumetric coverage, enhancing both geometric efficiency and simulation fidelity. Experiments demonstrate that PhysConvex achieves high-fidelity reconstruction of geometry, appearance, and physical properties from videos, outperforming existing methods.