Hierarchical Awareness Adapters with Hybrid Pyramid Feature Fusion for Dense Depth Prediction

TL;DR

This paper introduces a novel hierarchical model with hybrid pyramid feature fusion and a CRF decoder for improved monocular depth estimation, achieving state-of-the-art results efficiently.

Contribution

It proposes a multilevel perceptual CRF model with hybrid feature fusion and a hierarchical awareness adapter, enhancing depth prediction accuracy and computational efficiency.

Findings

Achieves state-of-the-art performance on NYU Depth v2 and KITTI datasets.

Reduces Abs Rel to 0.088 on NYU Depth v2.

Attains near-perfect threshold accuracy on KITTI with 194M parameters.

Abstract

Monocular depth estimation from a single RGB image remains a fundamental challenge in computer vision due to inherent scale ambiguity and the absence of explicit geometric cues. Existing approaches typically rely on increasingly complex network architectures to regress depth maps, which escalates training costs and computational overhead without fully exploiting inter-pixel spatial dependencies. We propose a multilevel perceptual conditional random field (CRF) model built upon the Swin Transformer backbone that addresses these limitations through three synergistic innovations: (1) an adaptive hybrid pyramid feature fusion (HPF) strategy that captures both short-range and long-range dependencies by combining multi-scale spatial pyramid pooling with biaxial feature aggregation, enabling effective integration of global and local contextual information; (2) a hierarchical awareness adapter (HA) that enriches cross-level feature interactions within the encoder through lightweight broadcast modules with learnable dimensional scaling, reducing computational complexity while enhancing representational capacity; and (3) a fully-connected CRF decoder with dynamic scaling attention that models fine-grained pixel-level spatial relationships, incorporating a bias learning unit to prevent extreme-value collapse and ensure stable training. Extensive experiments on NYU Depth v2, KITTI, and MatterPort3D datasets demonstrate that our method achieves state-of-the-art performance, reducing Abs Rel to 0.088 ($-$7.4\%) and RMSE to 0.316 ($-$5.4\%) on NYU Depth v2, while attaining near-perfect threshold accuracy ($\delta < 1.25^3 \approx 99.8\%$) on KITTI with only 194M parameters and 21ms inference time.