DeblurDiff: Real-World Image Deblurring with Generative Diffusion Models

TL;DR

DeblurDiff introduces a novel latent space approach using a co-trained kernel prediction network with diffusion models, significantly improving real-world image deblurring and detail preservation.

Contribution

The paper proposes a Latent Kernel Prediction Network co-trained with diffusion models, enabling robust, iterative, and structure-preserving real-world image deblurring.

Findings

Outperforms state-of-the-art deblurring methods on benchmarks.

Effectively preserves structural details in real-world images.

Enhances deblurring robustness through iterative kernel refinement.

Abstract

Diffusion models have achieved significant progress in image generation. The pre-trained Stable Diffusion (SD) models are helpful for image deblurring by providing clear image priors. However, directly using a blurry image or pre-deblurred one as a conditional control for SD will either hinder accurate structure extraction or make the results overly dependent on the deblurring network. In this work, we propose a Latent Kernel Prediction Network (LKPN) to achieve robust real-world image deblurring. Specifically, we co-train the LKPN in latent space with conditional diffusion. The LKPN learns a spatially variant kernel to guide the restoration of sharp images in the latent space. By applying element-wise adaptive convolution (EAC), the learned kernel is utilized to adaptively process the input feature, effectively preserving the structural information of the input. This process thereby more effectively guides the generative process of Stable Diffusion (SD), enhancing both the deblurring efficacy and the quality of detail reconstruction. Moreover, the results at each diffusion step are utilized to iteratively estimate the kernels in LKPN to better restore the sharp latent by EAC. This iterative refinement enhances the accuracy and robustness of the deblurring process. Extensive experimental results demonstrate that the proposed method outperforms state-of-the-art image deblurring methods on both benchmark and real-world images.