FreeGaussian: Annotation-free Control of Articulated Objects via 3D Gaussian Splats with Flow Derivatives

TL;DR

FreeGaussian introduces an annotation-free approach to reconstruct and control articulated objects from monocular video by disentangling motions using flow derivatives and a 3D Gaussian trajectory scheme, achieving state-of-the-art results.

Contribution

The paper presents a novel annotation-free method that leverages flow derivatives and 3D Gaussian trajectories for controllable articulated object reconstruction from monocular video.

Findings

Achieves state-of-the-art visual performance on articulated objects.

Provides precise, part-aware controllability without manual annotations.

Simplifies control signal computation with 3D Gaussian trajectories.

Abstract

Reconstructing controllable Gaussian splats for articulated objects from monocular video is especially challenging due to its inherently insufficient constraints. Existing methods address this by relying on dense masks and manually defined control signals, limiting their real-world applications. In this paper, we propose an annotation-free method, FreeGaussian, which mathematically disentangles camera egomotion and articulated movements via flow derivatives. By establishing a connection between 2D flows and 3D Gaussian dynamic flow, our method enables optimization and continuity of dynamic Gaussian motions from flow priors without any control signals. Furthermore, we introduce a 3D spherical vector controlling scheme, which represents the state as a 3D Gaussian trajectory, thereby eliminating the need for complex 1D control signal calculations and simplifying controllable Gaussian modeling. Extensive experiments on articulated objects demonstrate the state-of-the-art visual performance and precise, part-aware controllability of our method. Code is available at: https://github.com/Tavish9/freegaussian.