DDP: Diffusion Model for Dense Visual Prediction

TL;DR

This paper introduces DDP, a diffusion-based framework for dense visual prediction tasks that achieves state-of-the-art results without task-specific modifications, offering advantages like dynamic inference and uncertainty estimation.

Contribution

The paper presents DDP, a novel diffusion model framework that generalizes across multiple dense prediction tasks with high performance and simplicity, without requiring task-specific architecture changes.

Findings

Achieves 83.9 mIoU on Cityscapes for semantic segmentation

Attains 70.6 mIoU on nuScenes for BEV map segmentation

Reaches 0.05 REL in depth estimation on KITTI

Abstract

We propose a simple, efficient, yet powerful framework for dense visual predictions based on the conditional diffusion pipeline. Our approach follows a "noise-to-map" generative paradigm for prediction by progressively removing noise from a random Gaussian distribution, guided by the image. The method, called DDP, efficiently extends the denoising diffusion process into the modern perception pipeline. Without task-specific design and architecture customization, DDP is easy to generalize to most dense prediction tasks, e.g., semantic segmentation and depth estimation. In addition, DDP shows attractive properties such as dynamic inference and uncertainty awareness, in contrast to previous single-step discriminative methods. We show top results on three representative tasks with six diverse benchmarks, without tricks, DDP achieves state-of-the-art or competitive performance on each task compared to the specialist counterparts. For example, semantic segmentation (83.9 mIoU on Cityscapes), BEV map segmentation (70.6 mIoU on nuScenes), and depth estimation (0.05 REL on KITTI). We hope that our approach will serve as a solid baseline and facilitate future research