4C4D: 4 Camera 4D Gaussian Splatting

TL;DR

This paper introduces 4C4D, a novel framework for high-fidelity 4D Gaussian Splatting from videos captured by as few as four portable cameras, addressing the challenge of modeling scene dynamics with sparse views.

Contribution

The paper proposes a Neural Decaying Function to enhance geometric modeling in 4D Gaussian Splatting under sparse camera settings, improving performance over prior methods.

Findings

4C4D outperforms prior art on sparse-view datasets.

The Neural Decaying Function improves geometric learning in 4D Gaussian Splatting.

Extensive experiments validate the effectiveness of the proposed approach.

Abstract

This paper tackles the challenge of recovering 4D dynamic scenes from videos captured by as few as four portable cameras. Learning to model scene dynamics for temporally consistent novel-view rendering is a foundational task in computer graphics, where previous works often require dense multi-view captures using camera arrays of dozens or even hundreds of views. We propose \textbf{4C4D}, a novel framework that enables high-fidelity 4D Gaussian Splatting from video captures of extremely sparse cameras. Our key insight lies that the geometric learning under sparse settings is substantially more difficult than modeling appearance. Driven by this observation, we introduce a Neural Decaying Function on Gaussian opacities for enhancing the geometric modeling capability of 4D Gaussians. This design mitigates the inherent imbalance between geometry and appearance modeling in 4DGS by encouraging the 4DGS gradients to focus more on geometric learning. Extensive experiments across sparse-view datasets with varying camera overlaps show that 4C4D achieves superior performance over prior art. Project page at: https://junshengzhou.github.io/4C4D.