Blind Image Restoration via Fast Diffusion Inversion

TL;DR

This paper introduces BIRD, a novel blind image restoration method that jointly optimizes degradation parameters and images using a diffusion model, ensuring images stay on the data manifold and reducing computational costs.

Contribution

BIRD is the first blind IR method that optimizes degradation parameters and images simultaneously with a novel sampling technique on pre-trained diffusion models.

Findings

Achieves state-of-the-art results on multiple image restoration tasks.

Effectively models unknown degradations without altering the diffusion process.

Reduces computational cost by leveraging large time steps in diffusion inversion.

Abstract

Image Restoration (IR) methods based on a pre-trained diffusion model have demonstrated state-of-the-art performance. However, they have two fundamental limitations: 1) they often assume that the degradation operator is completely known and 2) they alter the diffusion sampling process, which may result in restored images that do not lie onto the data manifold. To address these issues, we propose Blind Image Restoration via fast Diffusion inversion (BIRD) a blind IR method that jointly optimizes for the degradation model parameters and the restored image. To ensure that the restored images lie onto the data manifold, we propose a novel sampling technique on a pre-trained diffusion model. A key idea in our method is not to modify the reverse sampling, i.e, not to alter all the intermediate latents, once an initial noise is sampled. This is ultimately equivalent to casting the IR task as an optimization problem in the space of the input noise. Moreover, to mitigate the computational cost associated with inverting a fully unrolled diffusion model, we leverage the inherent capability of these models to skip ahead in the forward diffusion process using large time steps. We experimentally validate BIRD on several image restoration tasks and show that it achieves state of the art performance on all of them. Our code is available at https://github.com/hamadichihaoui/BIRD.