FiffDepth: Feed-forward Transformation of Diffusion-Based Generators for Detailed Depth Estimation

TL;DR

FiffDepth introduces a novel feed-forward transformation of diffusion-based generators to enhance monocular depth estimation, achieving superior accuracy and detail while maintaining efficiency and robustness across diverse datasets.

Contribution

It transforms diffusion-based image generators into a feed-forward architecture for detailed depth estimation, combining generative features with strong generalization capabilities.

Findings

Achieves state-of-the-art accuracy in MDE tasks.

Demonstrates robustness and fine-grained detail in depth maps.

Outperforms existing methods on benchmark datasets.

Abstract

Monocular Depth Estimation (MDE) is a fundamental 3D vision problem with numerous applications such as 3D scene reconstruction, autonomous navigation, and AI content creation. However, robust and generalizable MDE remains challenging due to limited real-world labeled data and distribution gaps between synthetic datasets and real data. Existing methods often struggle with real-world test data with low efficiency, reduced accuracy, and lack of detail. To address these issues, we propose an efficient MDE approach named FiffDepth. The key feature of FiffDepth is its use of diffusion priors. It transforms diffusion-based image generators into a feed-forward architecture for detailed depth estimation. FiffDepth preserves key generative features and integrates the strong generalization capabilities of models like DINOv2. Through benchmark evaluations, we demonstrate that FiffDepth achieves exceptional accuracy, stability, and fine-grained detail, offering significant improvements in MDE performance against state-of-the-art MDE approaches. The paper's source code is available here: https://yunpeng1998.github.io/FiffDepth/