DPBridge: Latent Diffusion Bridge for Dense Prediction

TL;DR

DPBridge introduces a novel latent diffusion bridge framework that leverages structured visual priors and pretrained models to significantly improve dense prediction tasks like depth and surface normal estimation.

Contribution

It is the first to integrate diffusion bridge models with visual priors and pretrained diffusion backbones for dense prediction, offering a new effective approach.

Findings

Consistently outperforms existing methods on benchmark datasets.

Demonstrates strong generalization across different dense prediction tasks.

Effective integration of visual priors enhances prediction accuracy.

Abstract

Diffusion models demonstrate remarkable capabilities in capturing complex data distributions and have achieved compelling results in many generative tasks. While they have recently been extended to dense prediction tasks such as depth estimation and surface normal prediction, their full potential in this area remains underexplored. As target signal maps and input images are pixel-wise aligned, the conventional noise-to-data generation paradigm is inefficient, and input images can serve as a more informative prior compared to pure noise. Diffusion bridge models, which support data-to-data generation between two general data distributions, offer a promising alternative, but they typically fail to exploit the rich visual priors embedded in large pretrained foundation models. To address these limitations, we integrate diffusion bridge formulation with structured visual priors and introduce DPBridge, the first latent diffusion bridge framework for dense prediction tasks. To resolve the incompatibility between diffusion bridge models and pretrained diffusion backbones, we propose (1) a tractable reverse transition kernel for the diffusion bridge process, enabling maximum likelihood training scheme; (2) finetuning strategies including distribution-aligned normalization and image consistency loss. Experiments across extensive benchmarks validate that our method consistently achieves superior performance, demonstrating its effectiveness and generalization capability under different scenarios.