Blind Inverse Problem Solving Made Easy by Text-to-Image Latent Diffusion

TL;DR

LADiBI is a training-free, flexible method that uses text-to-image diffusion models within a Bayesian framework to solve diverse blind inverse image restoration problems without restrictive assumptions.

Contribution

Introducing LADiBI, a novel approach leveraging large-scale text-to-image diffusion for blind inverse problems, eliminating the need for retraining or restrictive assumptions.

Findings

Effective on both linear and nonlinear restoration tasks

Handles diverse image distributions without retraining

Operates without task-specific assumptions

Abstract

This paper considers blind inverse image restoration, the task of predicting a target image from a degraded source when the degradation (i.e. the forward operator) is unknown. Existing solutions typically rely on restrictive assumptions such as operator linearity, curated training data or narrow image distributions limiting their practicality. We introduce LADiBI, a training-free method leveraging large-scale text-to-image diffusion to solve diverse blind inverse problems with minimal assumptions. Within a Bayesian framework, LADiBI uses text prompts to jointly encode priors for both target images and operators, unlocking unprecedented flexibility compared to existing methods. Additionally, we propose a novel diffusion posterior sampling algorithm that combines strategic operator initialization with iterative refinement of image and operator parameters, eliminating the need for highly constrained operator forms. Experiments show that LADiBI effectively handles both linear and challenging nonlinear image restoration problems across various image distributions, all without task-specific assumptions or retraining.