OmniPhysGS: 3D Constitutive Gaussians for General Physics-Based Dynamics Generation

TL;DR

OmniPhysGS introduces a novel approach to generate realistic 3D dynamic scenes with diverse materials by modeling assets as collections of 3D Gaussians and leveraging a pretrained diffusion model for supervision.

Contribution

The paper presents OmniPhysGS, a flexible 3D scene synthesis method that models objects with constitutive Gaussians and multiple physical sub-models, enabling more general and plausible physics-based animations.

Findings

Achieves more realistic physical dynamics across various materials.

Surpasses existing methods by 3% to 16% in visual quality and text alignment.

Handles complex interactions between heterogeneous objects.

Abstract

Recently, significant advancements have been made in the reconstruction and generation of 3D assets, including static cases and those with physical interactions. To recover the physical properties of 3D assets, existing methods typically assume that all materials belong to a specific predefined category (e.g., elasticity). However, such assumptions ignore the complex composition of multiple heterogeneous objects in real scenarios and tend to render less physically plausible animation given a wider range of objects. We propose OmniPhysGS for synthesizing a physics-based 3D dynamic scene composed of more general objects. A key design of OmniPhysGS is treating each 3D asset as a collection of constitutive 3D Gaussians. For each Gaussian, its physical material is represented by an ensemble of 12 physical domain-expert sub-models (rubber, metal, honey, water, etc.), which greatly enhances the flexibility of the proposed model. In the implementation, we define a scene by user-specified prompts and supervise the estimation of material weighting factors via a pretrained video diffusion model. Comprehensive experiments demonstrate that OmniPhysGS achieves more general and realistic physical dynamics across a broader spectrum of materials, including elastic, viscoelastic, plastic, and fluid substances, as well as interactions between different materials. Our method surpasses existing methods by approximately 3% to 16% in metrics of visual quality and text alignment.