Variance-Reduced Diffusion Sampling via Target Score Identity

TL;DR

This paper introduces variance reduction techniques for diffusion sampling using the Target Score Identity, leading to improved sample quality in inverse problems and PDE applications.

Contribution

It develops a novel variance reduction framework based on the Target Score Identity, including importance sampling, blending rules, and extensions for inverse problems.

Findings

Enhanced sample quality in synthetic and PDE inverse problems

Effective variance reduction with the proposed methods

Compatibility with standard reverse-time solvers

Abstract

We study variance reduction for score estimation and diffusion-based sampling in settings where the clean (target) score is available or can be approximated. Starting from the Target Score Identity (TSI), which expresses the noisy marginal score as a conditional expectation of the target score under the forward diffusion, we develop: (i) a plug-and-play nonparametric self-normalized importance sampling estimator compatible with standard reverse-time solvers, (ii) a variance-minimizing \emph{state- and time-dependent} blending rule between Tweedie-type and TSI estimators together with an anti-correlation analysis, (iii) a data-only extension based on locally fitted proxy scores, and (iv) a likelihood-tilting extension to Bayesian inverse problems. We also propose a \emph{Critic--Gate} distillation scheme that amortizes the state-dependent blending coefficient into a neural gate. Experiments on synthetic targets and PDE-governed inverse problems demonstrate improved sample quality for a fixed simulation budget.