GemDepth: Geometry-Embedded Features for 3D-Consistent Video Depth

TL;DR

GemDepth introduces a geometry-aware framework for 3D-consistent video depth estimation, leveraging explicit camera motion and geometric embeddings to improve detail and temporal coherence.

Contribution

It proposes a novel Geometry-Embedding Module and an Alternating Spatio-Temporal Transformer to enhance 3D consistency and spatial detail in video depth estimation.

Findings

Achieves state-of-the-art performance on multiple datasets.

Effectively maintains 3D geometric consistency under view changes.

Improves spatial detail and temporal coherence simultaneously.

Abstract

Video depth estimation extends monocular prediction into the temporal domain to ensure coherence. However, existing methods often suffer from spatial blurring in fine-detail regions and temporal inconsistencies. We argue that current approaches, which primarily rely on temporal smoothing via Transformers, struggle to maintain strict 3D geometric consistency-particularly under rotations or drastic view changes. To address this, we propose GemDepth, a framework built on the insight that an explicit awareness of camera motion and global 3D structure is a prerequisite for 3D consistency. Distinctively, GemDepth introduces a Geometry-Embedding Module (GEM) that predicts inter-frame camera poses to generate implicit geometric embeddings. This injection of motion priors equips the network with intrinsic 3D perception and alignment capabilities. Guided by these geometric cues, our Alternating Spatio-Temporal Transformer (ASTT) captures latent point-level correspondences to simultaneously enhance spatial precision for sharp details and enforce rigorous temporal consistency. Furthermore, GemDepth employs a data-efficient training strategy, effectively bridging the gap between high efficiency and robust geometric consistency. As shown in Fig.2, comprehensive evaluations demonstrate that GemDepth achieves state-of-the-art performance across multiple datasets, particularly in complex dynamic scenarios. The code is publicly available at: https://github.com/Yuecheng919/GemDepth.