Text-to-3D Gaussian Splatting with Physics-Grounded Motion Generation

TL;DR

This paper introduces a novel text-to-3D generation framework that produces high-fidelity, physics-grounded 3D models with realistic motion, leveraging LLM-refined prompts, diffusion priors, and continuum mechanics-based deformation.

Contribution

It combines large language model prompts, diffusion priors, and physics-based deformation to generate 3D objects with realistic motion, addressing efficiency and physics simulation challenges.

Findings

Achieves high-quality, realistic 3D object generation.

Synthesizes physics-aware motion adhering to conservation laws.

Demonstrates superior performance in realistic physics-grounded animations.

Abstract

Text-to-3D generation is a valuable technology in virtual reality and digital content creation. While recent works have pushed the boundaries of text-to-3D generation, producing high-fidelity 3D objects with inefficient prompts and simulating their physics-grounded motion accurately still remain unsolved challenges. To address these challenges, we present an innovative framework that utilizes the Large Language Model (LLM)-refined prompts and diffusion priors-guided Gaussian Splatting (GS) for generating 3D models with accurate appearances and geometric structures. We also incorporate a continuum mechanics-based deformation map and color regularization to synthesize vivid physics-grounded motion for the generated 3D Gaussians, adhering to the conservation of mass and momentum. By integrating text-to-3D generation with physics-grounded motion synthesis, our framework renders photo-realistic 3D objects that exhibit physics-aware motion, accurately reflecting the behaviors of the objects under various forces and constraints across different materials. Extensive experiments demonstrate that our approach achieves high-quality 3D generations with realistic physics-grounded motion.