SparseCam4D: Spatio-Temporally Consistent 4D Reconstruction from Sparse Cameras

TL;DR

SparseCam4D introduces a novel framework for high-quality 4D dynamic scene reconstruction using sparse, uncalibrated cameras, leveraging a Spatio-Temporal Distortion Field to model inconsistencies and achieve consistent, high-fidelity renderings.

Contribution

The paper presents a new pipeline and the Spatio-Temporal Distortion Field for 4D reconstruction from sparse, uncalibrated cameras, improving consistency and quality over prior methods.

Findings

Achieves spatio-temporally consistent high-fidelity renderings.

Outperforms existing approaches on multi-camera dynamic scene benchmarks.

Effectively models inconsistencies in sparse, uncalibrated camera observations.

Abstract

High-quality 4D reconstruction enables photorealistic and immersive rendering of the dynamic real world. However, unlike static scenes that can be fully captured with a single camera, high-quality dynamic scenes typically require dense arrays of tens or even hundreds of synchronized cameras. Dependence on such costly lab setups severely limits practical scalability. To this end, we propose a sparse-camera dynamic reconstruction framework that exploits abundant yet inconsistent generative observations. Our key innovation is the Spatio-Temporal Distortion Field, which provides a unified mechanism for modeling inconsistencies in generative observations across both spatial and temporal dimensions. Building on this, we develop a complete pipeline that enables 4D reconstruction from sparse and uncalibrated camera inputs. We evaluate our method on multi-camera dynamic scene benchmarks, achieving spatio-temporally consistent high-fidelity renderings and significantly outperforming existing approaches. Project page available at https://inspatio.github.io/sparse-cam4d/