LD-RPS: Zero-Shot Unified Image Restoration via Latent Diffusion Recurrent Posterior Sampling

TL;DR

LD-RPS introduces a dataset-free, unified image restoration method using latent diffusion and recurrent posterior sampling, capable of handling various degradations without task-specific training.

Contribution

It presents a novel, task-blind approach leveraging pretrained latent diffusion models and recurrent refinement for unified image restoration.

Findings

Outperforms state-of-the-art methods in diverse restoration tasks.

Demonstrates robustness and generalizability across different degradations.

Operates without task-specific paired datasets.

Abstract

Unified image restoration is a significantly challenging task in low-level vision. Existing methods either make tailored designs for specific tasks, limiting their generalizability across various types of degradation, or rely on training with paired datasets, thereby suffering from closed-set constraints. To address these issues, we propose a novel, dataset-free, and unified approach through recurrent posterior sampling utilizing a pretrained latent diffusion model. Our method incorporates the multimodal understanding model to provide sematic priors for the generative model under a task-blind condition. Furthermore, it utilizes a lightweight module to align the degraded input with the generated preference of the diffusion model, and employs recurrent refinement for posterior sampling. Extensive experiments demonstrate that our method outperforms state-of-the-art methods, validating its effectiveness and robustness. Our code and data are available at https://github.com/AMAP-ML/LD-RPS.