ScaleDepth: Decomposing Metric Depth Estimation into Scale Prediction and Relative Depth Estimation

TL;DR

ScaleDepth introduces a novel approach to metric depth estimation by decomposing it into scene scale prediction and relative depth estimation, enabling better generalization across diverse scenes without fine-tuning.

Contribution

The paper proposes a new monocular depth estimation method that decomposes metric depth into scene scale and relative depth, with modules for semantic-aware scale prediction and adaptive relative depth estimation.

Findings

Achieves state-of-the-art results across indoor and outdoor scenes.

Effectively generalizes to unseen scenes without fine-tuning.

Handles diverse scene scales in a unified framework.

Abstract

Estimating depth from a single image is a challenging visual task. Compared to relative depth estimation, metric depth estimation attracts more attention due to its practical physical significance and critical applications in real-life scenarios. However, existing metric depth estimation methods are typically trained on specific datasets with similar scenes, facing challenges in generalizing across scenes with significant scale variations. To address this challenge, we propose a novel monocular depth estimation method called ScaleDepth. Our method decomposes metric depth into scene scale and relative depth, and predicts them through a semantic-aware scale prediction (SASP) module and an adaptive relative depth estimation (ARDE) module, respectively. The proposed ScaleDepth enjoys several merits. First, the SASP module can implicitly combine structural and semantic features of the images to predict precise scene scales. Second, the ARDE module can adaptively estimate the relative depth distribution of each image within a normalized depth space. Third, our method achieves metric depth estimation for both indoor and outdoor scenes in a unified framework, without the need for setting the depth range or fine-tuning model. Extensive experiments demonstrate that our method attains state-of-the-art performance across indoor, outdoor, unconstrained, and unseen scenes. Project page: https://ruijiezhu94.github.io/ScaleDepth