Frequency-Guided Posterior Sampling for Diffusion-Based Image Restoration

TL;DR

This paper analyzes the errors in current diffusion-based image restoration methods and proposes a frequency-guided sampling approach that improves results in tasks like deblurring and dehazing.

Contribution

It provides the first rigorous analysis of approximation errors in diffusion-based inverse problems and introduces a frequency-guided sampling method with an adaptive curriculum.

Findings

Significant reduction in restoration errors for motion deblurring.

Improved image quality in dehazing tasks.

Theoretical insights guide effective frequency domain modifications.

Abstract

Image restoration aims to recover high-quality images from degraded observations. When the degradation process is known, the recovery problem can be formulated as an inverse problem, and in a Bayesian context, the goal is to sample a clean reconstruction given the degraded observation. Recently, modern pretrained diffusion models have been used for image restoration by modifying their sampling procedure to account for the degradation process. However, these methods often rely on certain approximations that can lead to significant errors and compromised sample quality. In this paper, we provide the first rigorous analysis of this approximation error for linear inverse problems under distributional assumptions on the space of natural images, demonstrating cases where previous works can fail dramatically. Motivated by our theoretical insights, we propose a simple modification to existing diffusion-based restoration methods. Our approach introduces a time-varying low-pass filter in the frequency domain of the measurements, progressively incorporating higher frequencies during the restoration process. We develop an adaptive curriculum for this frequency schedule based on the underlying data distribution. Our method significantly improves performance on challenging image restoration tasks including motion deblurring and image dehazing.