InfoGaussian: Structure-Aware Dynamic Gaussians through Lightweight Information Shaping

TL;DR

This paper introduces InfoGaussian, a structure-aware method for controlling dynamic 3D Gaussian scene representations by enforcing movement resonance through mutual information, enabling efficient and consistent object animation.

Contribution

It proposes a mutual information shaping technique with lightweight optimization to achieve structured, dynamic Gaussian scene representations that reflect underlying object movements.

Findings

Improves consistency of object movements in 3D Gaussian scenes

Enhances 3D object segmentation accuracy

Reduces computational and memory costs

Abstract

3D Gaussians, as a low-level scene representation, typically involve thousands to millions of Gaussians. This makes it difficult to control the scene in ways that reflect the underlying dynamic structure, where the number of independent entities is typically much smaller. In particular, it can be challenging to animate and move objects in the scene, which requires coordination among many Gaussians. To address this issue, we develop a mutual information shaping technique that enforces movement resonance between correlated Gaussians in a motion network. Such correlations can be learned from putative 2D object masks in different views. By approximating the mutual information with the Jacobians of the motions, our method ensures consistent movements of the Gaussians composing different objects under various perturbations. In particular, we develop an efficient contrastive training pipeline with lightweight optimization to shape the motion network, avoiding the need for re-shaping throughout the motion sequence. Notably, our training only touches a small fraction of all Gaussians in the scene yet attains the desired compositional behavior according to the underlying dynamic structure. The proposed technique is evaluated on challenging scenes and demonstrates significant performance improvement in promoting consistent movements and 3D object segmentation while inducing low computation and memory requirements.