AnchorD: Metric Grounding of Monocular Depth Using Factor Graphs

TL;DR

This paper introduces AnchorD, a training-free framework that improves monocular depth estimation by grounding predictions in real-world metrics using factor graph optimization, especially on challenging surfaces.

Contribution

It presents a novel depth grounding method that aligns monocular depth predictions with sensor data without retraining, and provides a new benchmark dataset for evaluation.

Findings

Consistent depth accuracy improvements across various sensors.

Effective grounding on non-Lambertian and reflective surfaces.

No additional training required for the depth correction.

Abstract

Dense and accurate depth estimation is essential for robotic manipulation, grasping, and navigation, yet currently available depth sensors are prone to errors on transparent, specular, and general non-Lambertian surfaces. To mitigate these errors, large-scale monocular depth estimation approaches provide strong structural priors, but their predictions can be potentially skewed or mis-scaled in metric units, limiting their direct use in robotics. Thus, in this work, we propose a training-free depth grounding framework that anchors monocular depth estimation priors from a depth foundation model in raw sensor depth through factor graph optimization. Our method performs a patch-wise affine alignment, locally grounding monocular predictions in metric real-world depth while preserving fine-grained geometric structure and discontinuities. To facilitate evaluation in challenging real-world conditions, we introduce a benchmark dataset with dense scene-wide ground truth depth in the presence of non-Lambertian objects. Ground truth is obtained via matte reflection spray and multi-camera fusion, overcoming the reliance on object-only CAD-based annotations used in prior datasets. Extensive evaluations across diverse sensors and domains demonstrate consistent improvements in depth performance without any (re-)training. We make our implementation publicly available at https://anchord.cs.uni-freiburg.de.