Last-Layer-Centric Feature Recombination: Unleashing 3D Geometric Knowledge in DINOv3 for Monocular Depth Estimation

TL;DR

This paper analyzes DINOv3's layers for monocular depth estimation, revealing non-uniform distribution of 3D info, and introduces a Last-Layer-Centric Feature Recombination module that improves accuracy.

Contribution

It uncovers the non-uniform distribution of 3D geometric knowledge in DINOv3 layers and proposes a novel feature recombination method to enhance depth estimation performance.

Findings

Deeper layers in DINOv3 have stronger depth predictability.

The proposed LFR module improves monocular depth estimation accuracy.

LFR achieves state-of-the-art performance on benchmark datasets.

Abstract

Monocular depth estimation (MDE) is a fundamental yet inherently ill-posed task. Recent vision foundation models (VFMs), particularly DINO-based transformers, have significantly improved accuracy and generalization for dense prediction. Prior works generally follow a unified paradigm: sampling a fixed set of intermediate transformer layers at uniform intervals to build multi-scale features. This common practice implicitly assumes that geometric information is uniformly distributed across layers, which may underutilize the structural 3D cues encoded in VFMs. In this study, we present a systematic layer-wise analysis of DINOv3, revealing that 3D information is distributed non-uniformly: deeper layers exhibit stronger depth predictability and better capture inter-sample geometric variation. Motivated by this, we introduce a Last-Layer-Centric Feature Recombination (LFR) module to enhance geometric expressiveness. LFR treats the final layer as a geometric anchor and adaptively selects complementary intermediate layers according to a minimal-similarity criterion. Selected features are fused with the last-layer representation via compact linear adapters.Extensive experiments show that LFR module consistently improves MDE accuracy and achieves state-of-the-art performance. Our analysis sheds light on how geometric knowledge is organized within VFMs and offers an efficient strategy for unlocking their potential in dense 3D tasks.