Complet4R: Geometric Complete 4D Reconstruction

TL;DR

Complet4R is an end-to-end transformer-based framework that achieves state-of-the-art geometric 4D scene reconstruction and 3D point tracking from video sequences, including occluded regions.

Contribution

It introduces a unified approach for complete 4D reconstruction using a decoder-only transformer that globally processes sequential video data.

Findings

State-of-the-art performance on Geometric Complete 4D Reconstruction benchmark.

Effective reconstruction of occluded regions in dynamic scenes.

Improved 3D point tracking accuracy.

Abstract

We introduce Complet4R, a novel end-to-end framework for Geometric Complete 4D Reconstruction, which aims to recover temporally coherent and geometrically complete reconstruction for dynamic scenes. Our method formalizes the task of Geometric Complete 4D Reconstruction as a unified framework of reconstruction and completion, by directly accumulating full contexts onto each frame. Unlike previous approaches that rely on pairwise reconstruction or local motion estimation, Complet4R utilizes a decoder-only transformer to operate all context globally directly from sequential video input, reconstructing a complete geometry for every single timestamp, including occluded regions visible in other frames. Our method demonstrates the state-of-the-art performance on our proposed benchmark for Geometric Complete 4D Reconstruction and the 3D Point Tracking task. Code will be released to support future research.