The Midas Touch for Metric Depth

TL;DR

The paper introduces MTD, a method that converts relative depth to metric depth using sparse 3D data, improving accuracy, consistency, and efficiency for 3D applications.

Contribution

It presents a mathematically interpretable approach combining sparse graph optimization and discontinuity-aware refinement to enhance depth estimation.

Findings

MTD achieves substantial accuracy improvements over previous methods.

It exhibits strong generalization across different scenes.

The lightweight design facilitates deployment in diverse 3D tasks.

Abstract

Recent advances have markedly improved the cross-scene generalization of relative depth estimation, yet its practical applicability remains limited by the absence of metric scale, local inconsistencies, and low computational efficiency. To address these issues, we present \emph{\textbf{M}idas \textbf{T}ouch for \textbf{D}epth} (MTD), a mathematically interpretable approach that converts relative depth into metric depth using only extremely sparse 3D data. To eliminate local scale inconsistencies, it applies a segment-wise recovery strategy via sparse graph optimization, followed by a pixel-wise refinement strategy using a discontinuity-aware geodesic cost. MTD exhibits strong generalization and achieves substantial accuracy improvements over previous depth completion and depth estimation methods. Moreover, its lightweight, plug-and-play design facilitates deployment and integration on diverse downstream 3D tasks. Project page is available at https://mias.group/MTD.