Video Depth Propagation

TL;DR

VeloDepth is an efficient online video depth estimation method that leverages spatiotemporal priors and feature propagation to achieve real-time, consistent, and accurate depth predictions across video frames.

Contribution

The paper introduces VeloDepth, a novel propagation module and a structurally enforced temporal consistency approach for real-time video depth estimation.

Findings

Achieves state-of-the-art temporal consistency on benchmarks.

Provides faster inference compared to existing methods.

Maintains competitive depth accuracy in real-time applications.

Abstract

Depth estimation in videos is essential for visual perception in real-world applications. However, existing methods either rely on simple frame-by-frame monocular models, leading to temporal inconsistencies and inaccuracies, or use computationally demanding temporal modeling, unsuitable for real-time applications. These limitations significantly restrict general applicability and performance in practical settings. To address this, we propose VeloDepth, an efficient and robust online video depth estimation pipeline that effectively leverages spatiotemporal priors from previous depth predictions and performs deep feature propagation. Our method introduces a novel Propagation Module that refines and propagates depth features and predictions using flow-based warping coupled with learned residual corrections. In addition, our design structurally enforces temporal consistency, resulting in stable depth predictions across consecutive frames with improved efficiency. Comprehensive zero-shot evaluation on multiple benchmarks demonstrates the state-of-the-art temporal consistency and competitive accuracy of VeloDepth, alongside its significantly faster inference compared to existing video-based depth estimators. VeloDepth thus provides a practical, efficient, and accurate solution for real-time depth estimation suitable for diverse perception tasks. Code and models are available at https://github.com/lpiccinelli-eth/velodepth