DepthMaster: Taming Diffusion Models for Monocular Depth Estimation

TL;DR

DepthMaster introduces a single-step diffusion model with feature alignment and Fourier enhancement modules for improved monocular depth estimation, balancing global structure and fine details.

Contribution

It proposes a novel single-step diffusion approach with modules for semantic feature alignment and Fourier-based detail enhancement, advancing depth estimation accuracy and efficiency.

Findings

Achieves state-of-the-art generalization in depth estimation.

Effectively balances low-frequency structure and high-frequency details.

Outperforms existing diffusion-based methods across datasets.

Abstract

Monocular depth estimation within the diffusion-denoising paradigm demonstrates impressive generalization ability but suffers from low inference speed. Recent methods adopt a single-step deterministic paradigm to improve inference efficiency while maintaining comparable performance. However, they overlook the gap between generative and discriminative features, leading to suboptimal results. In this work, we propose DepthMaster, a single-step diffusion model designed to adapt generative features for the discriminative depth estimation task. First, to mitigate overfitting to texture details introduced by generative features, we propose a Feature Alignment module, which incorporates high-quality semantic features to enhance the denoising network's representation capability. Second, to address the lack of fine-grained details in the single-step deterministic framework, we propose a Fourier Enhancement module to adaptively balance low-frequency structure and high-frequency details. We adopt a two-stage training strategy to fully leverage the potential of the two modules. In the first stage, we focus on learning the global scene structure with the Feature Alignment module, while in the second stage, we exploit the Fourier Enhancement module to improve the visual quality. Through these efforts, our model achieves state-of-the-art performance in terms of generalization and detail preservation, outperforming other diffusion-based methods across various datasets. Our project page can be found at https://indu1ge.github.io/DepthMaster_page.