4DVGGT-D: 4D Visual Geometry Transformer with Improved Dynamic Depth Estimation

TL;DR

This paper introduces a novel, training-free framework for 4D scene reconstruction from monocular videos that effectively disentangles dynamic and static elements, leading to improved geometric accuracy.

Contribution

The proposed method offers a new decoupling approach with three components that stabilize camera pose, decompose depth manifolds, and adaptively fuse predictions without fine-tuning.

Findings

Achieves consistent improvements on 4D reconstruction benchmarks.

Performs competitively without requiring fine-tuning.

Demonstrates effective dynamic-static disentanglement in complex scenes.

Abstract

Reconstructing dynamic 4D scenes from monocular videos is a fundamental yet challenging task. While recent 3D foundation models provide strong geometric priors, their performance significantly degrades in dynamic environments. This degradation stems from a fundamental tension: the inherent coupling of camera ego-motion and object motion within global attention mechanisms. In this paper, we propose a novel, training-free progressive decoupling framework that disentangles dynamics from statics in a principled, coarse-to-fine manner. Our core insight is to resolve the tension by first stabilizing the camera pose, followed by geometric refinement. Specifically, our approach consists of three synergistic components: (1) a Dynamic-Mask-Guided Pose Decoupling module that isolates pose estimation from dynamic interference, yielding a stable motion-free reference frame; (2) a Topological Subspace Surgery mechanism that orthogonally decomposes the depth manifold, safely preserving dynamic objects while injecting refined, mask-aware geometry into static regions; and (3) an Information-Theoretic Confidence-Aware Fusion strategy that formulates depth integration as a heteroscedastic Bayesian inference problem, adaptively blending multi-pass predictions via inverse-variance weighting. Extensive experiments on standard 4D reconstruction benchmarks demonstrate that our method achieves consistent and substantial improvements across principal point-cloud metrics. Notably, our approach shows competitive performance in robust 4D scene reconstruction without requiring fine-tuning, suggesting the potential of mathematically grounded dynamic-static disentanglement.