# ER-LoRA: Effective-Rank Guided Adaptation for Weather-Generalized Depth Estimation

## TL;DR

This paper introduces ER-LoRA, a PEFT approach using effective-rank guided adaptation for weather-generalized depth estimation, leveraging pretrained vision models with minimal high-visibility data to outperform existing methods.

## Contribution

The paper proposes a novel STM strategy for PEFT that decomposes pretrained weights based on effective ranks, enabling flexible adaptation and strong generalization in depth estimation under adverse weather.

## Key findings

- STM outperforms existing PEFT and full fine-tuning methods.
- Our approach surpasses synthetic data trained methods.
- Achieves state-of-the-art results across multiple weather conditions.

## Abstract

Monocular depth estimation under adverse weather conditions (e.g.\ rain, fog, snow, and nighttime) remains highly challenging due to the lack of reliable ground truth and the difficulty of learning from unlabeled real-world data. Existing methods often rely on synthetic adverse data with pseudo-labels, which suffer from domain gaps, or employ self-supervised learning, which violates photometric assumptions in adverse scenarios. In this work, we propose to achieve weather-generalized depth estimation by Parameter-Efficient Fine-Tuning (PEFT) of Vision Foundation Models (VFMs), using only a small amount of high-visibility (normal) data. While PEFT has shown strong performance in semantic tasks such as segmentation, it remains underexplored for geometry -- centric tasks like depth estimation -- especially in terms of balancing effective adaptation with the preservation of pretrained knowledge. To this end, we introduce the Selecting-Tuning-Maintaining (STM) strategy, which structurally decomposes the pretrained weights of VFMs based on two kinds of effective ranks (entropy-rank and stable-rank). In the tuning phase, we adaptively select the proper rank number as well as the task-aware singular directions for initialization, based on the entropy-rank and full-tuned weight; while in the maintaining stage, we enforce a principal direction regularization based on the stable-rank. This design guarantees flexible task adaptation while preserving the strong generalization capability of the pretrained VFM. Extensive experiments on four real-world benchmarks across diverse weather conditions demonstrate that STM not only outperforms existing PEFT methods and full fine-tuning but also surpasses methods trained with adverse synthetic data, and even the depth foundation model