Enhancing Generalization of Depth Estimation Foundation Model via Weakly-Supervised Adaptation with Regularization

TL;DR

This paper introduces WeSTAR, a novel weakly-supervised adaptation framework that enhances the robustness and generalization of depth estimation foundation models across diverse and unseen domains.

Contribution

We propose a parameter-efficient adaptation method combining dense self-training, semantically-aware normalization, and weak supervision with regularization to improve depth estimation models.

Findings

WeSTAR outperforms existing methods on various out-of-distribution datasets.

The framework achieves state-of-the-art results across multiple benchmarks.

It effectively mitigates topological errors and maintains model stability.

Abstract

The emergence of foundation models has substantially advanced zero-shot generalization in monocular depth estimation (MDE), as exemplified by the Depth Anything series. However, given access to some data from downstream tasks, a natural question arises: can the performance of these models be further improved? To this end, we propose WeSTAR, a parameter-efficient framework that performs Weakly supervised Self-Training Adaptation with Regularization, designed to enhance the robustness of MDE foundation models in unseen and diverse domains. We first adopt a dense self-training objective as the primary source of structural self-supervision. To further improve robustness, we introduce semantically-aware hierarchical normalization, which exploits instance-level segmentation maps to perform more stable and multi-scale structural normalization. Beyond dense supervision, we introduce a cost-efficient weak supervision in the form of pairwise ordinal depth annotations to further guide the adaptation process, which enforces informative ordinal constraints to mitigate local topological errors. Finally, a weight regularization loss is employed to anchor the LoRA updates, ensuring training stability and preserving the model's generalizable knowledge. Extensive experiments on both realistic and corrupted out-of-distribution datasets under diverse and challenging scenarios demonstrate that WeSTAR consistently improves generalization and achieves state-of-the-art performance across a wide range of benchmarks.