Towards Zero-Shot Scale-Aware Monocular Depth Estimation

TL;DR

ZeroDepth introduces a novel monocular depth estimation framework that predicts metric scale across diverse domains by leveraging geometric embeddings and a decoupled encoder-decoder architecture, achieving state-of-the-art results.

Contribution

The paper presents ZeroDepth, a new approach that enables zero-shot, scale-aware monocular depth estimation across multiple domains without test-time scaling.

Findings

Achieves state-of-the-art performance on outdoor benchmarks like KITTI and nuScenes.

Outperforms in-domain trained methods requiring test-time scaling.

Successfully generalizes to indoor datasets like NYUv2 without domain-specific tuning.

Abstract

Monocular depth estimation is scale-ambiguous, and thus requires scale supervision to produce metric predictions. Even so, the resulting models will be geometry-specific, with learned scales that cannot be directly transferred across domains. Because of that, recent works focus instead on relative depth, eschewing scale in favor of improved up-to-scale zero-shot transfer. In this work we introduce ZeroDepth, a novel monocular depth estimation framework capable of predicting metric scale for arbitrary test images from different domains and camera parameters. This is achieved by (i) the use of input-level geometric embeddings that enable the network to learn a scale prior over objects; and (ii) decoupling the encoder and decoder stages, via a variational latent representation that is conditioned on single frame information. We evaluated ZeroDepth targeting both outdoor (KITTI, DDAD, nuScenes) and indoor (NYUv2) benchmarks, and achieved a new state-of-the-art in both settings using the same pre-trained model, outperforming methods that train on in-domain data and require test-time scaling to produce metric estimates.