Learning Deblurring Texture Prior from Unpaired Data with Diffusion Model

TL;DR

This paper introduces a diffusion model-based framework for blind image deblurring that learns spatially varying texture priors from unpaired data, avoiding adversarial training and effectively restoring textures.

Contribution

It proposes a novel diffusion model framework with a Texture Prior Encoder and Texture Transfer Transformer layer for unsupervised deblurring from unpaired data.

Findings

Outperforms state-of-the-art methods on benchmark datasets.

Effectively restores high-frequency texture details.

Provides a practical unsupervised deblurring solution.

Abstract

Since acquiring large amounts of realistic blurry-sharp image pairs is difficult and expensive, learning blind image deblurring from unpaired data is a more practical and promising solution. Unfortunately, dominant approaches rely heavily on adversarial learning to bridge the gap from blurry domains to sharp domains, ignoring the complex and unpredictable nature of real-world blur patterns. In this paper, we propose a novel diffusion model (DM)-based framework, dubbed \ours, for image deblurring by learning spatially varying texture prior from unpaired data. In particular, \ours performs DM to generate the prior knowledge that aids in recovering the textures of blurry images. To implement this, we propose a Texture Prior Encoder (TPE) that introduces a memory mechanism to represent the image textures and provides supervision for DM training. To fully exploit the generated texture priors, we present the Texture Transfer Transformer layer (TTformer), in which a novel Filter-Modulated Multi-head Self-Attention (FM-MSA) efficiently removes spatially varying blurring through adaptive filtering. Furthermore, we implement a wavelet-based adversarial loss to preserve high-frequency texture details. Extensive evaluations show that \ours provides a promising unsupervised deblurring solution and outperforms SOTA methods in widely-used benchmarks.