Visual-Instructed Degradation Diffusion for All-in-One Image Restoration

TL;DR

Defusion introduces a unified diffusion-based framework guided by visual instructions for versatile image restoration, effectively handling multiple degradation types and outperforming existing methods in diverse real-world scenarios.

Contribution

It presents a novel visual instruction-guided diffusion approach that explicitly models degradation patterns for all-in-one image restoration.

Findings

Outperforms state-of-the-art methods across various restoration tasks

Handles complex and real-world degradations effectively

Provides a stable and generalizable restoration process

Abstract

Image restoration tasks like deblurring, denoising, and dehazing usually need distinct models for each degradation type, restricting their generalization in real-world scenarios with mixed or unknown degradations. In this work, we propose \textbf{Defusion}, a novel all-in-one image restoration framework that utilizes visual instruction-guided degradation diffusion. Unlike existing methods that rely on task-specific models or ambiguous text-based priors, Defusion constructs explicit \textbf{visual instructions} that align with the visual degradation patterns. These instructions are grounded by applying degradations to standardized visual elements, capturing intrinsic degradation features while agnostic to image semantics. Defusion then uses these visual instructions to guide a diffusion-based model that operates directly in the degradation space, where it reconstructs high-quality images by denoising the degradation effects with enhanced stability and generalizability. Comprehensive experiments demonstrate that Defusion outperforms state-of-the-art methods across diverse image restoration tasks, including complex and real-world degradations.