CoGS: Controllable Gaussian Splatting

TL;DR

CoGS introduces a real-time, controllable Gaussian Splatting method for dynamic 3D scene reconstruction that outperforms existing neural representations in visual fidelity without requiring extensive calibration or pre-computed control signals.

Contribution

This paper presents CoGS, a novel method enabling real-time controllable Gaussian Splatting for dynamic scenes, eliminating the need for synchronized multi-view cameras and pre-computed control signals.

Findings

Outperforms existing methods in visual fidelity

Works on both synthetic and real-world datasets

Enables real-time control of dynamic scenes

Abstract

Capturing and re-animating the 3D structure of articulated objects present significant barriers. On one hand, methods requiring extensively calibrated multi-view setups are prohibitively complex and resource-intensive, limiting their practical applicability. On the other hand, while single-camera Neural Radiance Fields (NeRFs) offer a more streamlined approach, they have excessive training and rendering costs. 3D Gaussian Splatting would be a suitable alternative but for two reasons. Firstly, existing methods for 3D dynamic Gaussians require synchronized multi-view cameras, and secondly, the lack of controllability in dynamic scenarios. We present CoGS, a method for Controllable Gaussian Splatting, that enables the direct manipulation of scene elements, offering real-time control of dynamic scenes without the prerequisite of pre-computing control signals. We evaluated CoGS using both synthetic and real-world datasets that include dynamic objects that differ in degree of difficulty. In our evaluations, CoGS consistently outperformed existing dynamic and controllable neural representations in terms of visual fidelity.