Diffusion Restoration Adapter for Real-World Image Restoration

TL;DR

This paper introduces a lightweight diffusion model adapter that enhances real-world image restoration by leveraging pretrained priors, achieving high-quality results with fewer parameters compared to existing methods.

Contribution

A novel lightweight Adapter for diffusion models that improves image restoration efficiency and quality without large network copying, adaptable to various architectures.

Findings

Achieves photo-realistic image restoration with fewer parameters.

Compatible with denoising UNet and DiT architectures.

Outperforms existing methods in quality and efficiency.

Abstract

Diffusion models have demonstrated their powerful image generation capabilities, effectively fitting highly complex image distributions. These models can serve as strong priors for image restoration. Existing methods often utilize techniques like ControlNet to sample high quality images with low quality images from these priors. However, ControlNet typically involves copying a large part of the original network, resulting in a significantly large number of parameters as the prior scales up. In this paper, we propose a relatively lightweight Adapter that leverages the powerful generative capabilities of pretrained priors to achieve photo-realistic image restoration. The Adapters can be adapt to both denoising UNet and DiT, and performs excellent.