Syn4D: A Multiview Synthetic 4D Dataset

TL;DR

Syn4D is a comprehensive synthetic multiview dataset designed to advance 4D scene reconstruction, tracking, and human pose estimation in dynamic scenes from monocular video.

Contribution

It provides high-quality ground-truth annotations including camera motion, depth, and human pose, enabling improved research in dynamic scene understanding.

Findings

Demonstrated effectiveness of Syn4D in 4D reconstruction and tracking tasks.

Enabled geometry-aware camera retargeting and pose estimation with synthetic data.

Showcased potential to accelerate research in dynamic scene analysis.

Abstract

Dense 3D reconstruction and tracking of dynamic scenes from monocular video remains an important open challenge in computer vision. Progress in this area has been constrained by the scarcity of high-quality datasets with dense, complete, and accurate geometric annotations. To address this limitation, we introduce Syn4D, a multiview synthetic dataset of dynamic scenes that includes ground-truth camera motion, depth maps, dense tracking, and parametric human pose annotations. A key feature of Syn4D is the ability to unproject any pixel into 3D to any time and to any camera. We conduct extensive evaluations across multiple downstream tasks to demonstrate the utility and effectiveness of the proposed dataset, including 4D scene reconstruction, 3D point tracking, geometry-aware camera retargeting, and human pose estimation. The experimental results highlight Syn4D's potential to facilitate research in dynamic scene understanding and spatiotemporal modeling.