Inverse problems with diffusion models: MAP estimation via mode-seeking loss

TL;DR

This paper introduces a new MAP estimation method called VML for inverse problems using diffusion models, which guides samples towards modes efficiently without task-specific training.

Contribution

The paper proposes VML, a novel loss function derived from KL divergence minimization, enabling effective MAP estimation with diffusion models without approximations.

Findings

VML effectively guides diffusion models to MAP estimates.

VML-MAP outperforms existing methods in image restoration tasks.

The approach reduces computational time for inverse problem solving.

Abstract

A pre-trained unconditional diffusion model, combined with posterior sampling or maximum a posteriori (MAP) estimation techniques, can solve arbitrary inverse problems without task-specific training or fine-tuning. However, existing posterior sampling and MAP estimation methods often rely on modeling approximations and can also be computationally demanding. In this work, we propose a new MAP estimation strategy for solving inverse problems with a pre-trained unconditional diffusion model. Specifically, we introduce the variational mode-seeking loss (VML) and show that its minimization at each reverse diffusion step guides the generated sample towards the MAP estimate (modes in practice). VML arises from a novel perspective of minimizing the Kullback-Leibler (KL) divergence between the diffusion posterior $p(\mathbf{x}_0|\mathbf{x}_t)$ and the measurement posterior $p(\mathbf{x}_0|\mathbf{y})$, where $\mathbf{y}$ denotes the measurement. Importantly, for linear inverse problems, VML can be analytically derived without any modeling approximations. Based on further theoretical insights, we propose VML-MAP, an empirically effective algorithm for solving inverse problems via VML minimization, and validate its efficacy in both performance and computational time through extensive experiments on diverse image-restoration tasks across multiple datasets.