AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion

TL;DR

AutoDIR is an all-in-one image restoration system that automatically detects and restores various unknown degradations using latent diffusion, enabling flexible editing and superior performance across multiple tasks.

Contribution

AutoDIR introduces a novel two-stage framework combining blind quality assessment and latent diffusion for versatile, automatic image restoration and editing.

Findings

Outperforms state-of-the-art methods on diverse restoration tasks.

Enables user-controlled, open-vocabulary image editing.

Generalizes well to new degradation types and tasks.

Abstract

We present AutoDIR, an innovative all-in-one image restoration system incorporating latent diffusion. AutoDIR excels in its ability to automatically identify and restore images suffering from a range of unknown degradations. AutoDIR offers intuitive open-vocabulary image editing, empowering users to customize and enhance images according to their preferences. Specifically, AutoDIR consists of two key stages: a Blind Image Quality Assessment (BIQA) stage based on a semantic-agnostic vision-language model which automatically detects unknown image degradations for input images, an All-in-One Image Restoration (AIR) stage utilizes structural-corrected latent diffusion which handles multiple types of image degradations. Extensive experimental evaluation demonstrates that AutoDIR outperforms state-of-the-art approaches for a wider range of image restoration tasks. The design of AutoDIR also enables flexible user control (via text prompt) and generalization to new tasks as a foundation model of image restoration. Project is available at: \url{https://jiangyitong.github.io/AutoDIR_webpage/}.