PG-3DGS: Optimizing 3D Gaussian Splatting to Satisfy Physics Objectives

TL;DR

PG-3DGS introduces a framework that integrates differentiable physics simulation with 3D Gaussian splatting to generate geometries that are both visually accurate and physically functional.

Contribution

It couples physics simulation with 3D Gaussian representations, enabling shape optimization guided by physical objectives alongside visual losses.

Findings

Improves physical functionality of generated 3D structures.

Produces geometries that can perform tasks like pouring and generating lift.

Demonstrates higher lift in physical tests with 3D-printed models.

Abstract

Recent advances in Gaussian Splatting have enabled fast, high-fidelity 3D scene generation, yet these methods remain purely visual and lack an understanding of how shapes behave in the physical world. We introduce Physics-Guided 3D Gaussian Splatting (PG-3DGS), a framework that couples differentiable physics simulation with 3D Gaussian representations to generate 3D structures satisfying physics functionalities. By allowing physical objectives to guide the shape optimization process alongside visual losses, our approach produces geometries that are not only photometrically accurate but also physically functional. The model learns to adjust shapes so that the generated objects exhibit physically meaningful behaviors, for example, teapots that can pour and airplanes that can generate lift, without sacrificing visual quality. Experiments on pouring and aerodynamic lift tasks show that PG-3DGS improves physical functionality while preserving visual quality. In addition to simulation gains, bench-top physical lift tests with 3D-printed aircraft (Cessna, B-2 Spirit, and paper plane) under identical airflow conditions show higher scale-measured lift for PG-3DGS, generated structures than an appearance-matching baseline in all three cases. Our unified framework connects appearance-based reconstruction with physics-based reasoning, enabling end-to-end generation of 3D structures that both look realistic and function correctly.