Conditional Diffusion Under Linear Constraints: Langevin Mixing and Information-Theoretic Guarantees

TL;DR

This paper develops a theoretically grounded method for zero-shot conditional sampling with diffusion models in linear inverse problems, improving sampling accuracy by addressing score function biases.

Contribution

It introduces an information-theoretic analysis of score function errors and proposes a new sampling method that outperforms existing projection-based approaches.

Findings

The proposed method achieves better inpainting and super-resolution results.

Error bounds are derived based on conditional mutual information.

The approach outperforms strong baselines in experiments.

Abstract

We study zero-shot conditional sampling with pretrained diffusion models for linear inverse problems, including inpainting and super-resolution. In these problems, the observation determines only part of the unknown signal. The remaining degrees of freedom must be sampled according to the correct conditional data distribution. Existing projection-based samplers enforce measurement consistency by correcting the observed component during reverse diffusion. However, measurement consistency alone does not determine how probability mass should be distributed along the feasible set, and this can lead to biased conditional samples. We analyze this issue through a normal--tangent decomposition of the score function. For Gaussian noising, the observed-direction score is exactly determined by the measurement; only the tangent conditional score is unknown. We prove that the error from replacing this score by the unconditional tangent score is upper bounded by a dimension-free conditional mutual information between observed and unobserved components. This gives an information-theoretic decomposition into initialization and pathwise score-mismatch errors. Motivated by the theory, we propose a projected-Langevin initialization followed by guided reverse denoising, which outperforms a strong projection-based baseline in inpainting and super-resolution experiments.