Control Variate Score Matching for Diffusion Models

TL;DR

This paper introduces the Control Variate Score Identity (CVSI), a novel method that unifies existing score estimators for diffusion models, significantly reducing variance and improving sample efficiency across different noise levels.

Contribution

The paper proposes CVSI, a new variance reduction technique for score estimation in diffusion models, unifying DSI and TSI within a control variate framework with an optimal, time-dependent coefficient.

Findings

CVSI reduces variance in score estimation across noise levels.

Enhanced sample efficiency in diffusion model sampling.

Theoretical guarantee of variance minimization.

Abstract

Diffusion models offer a robust framework for sampling from unnormalized probability densities, which requires accurately estimating the score of the noise-perturbed target distribution. While the standard Denoising Score Identity (DSI) relies on data samples, access to the target energy function enables an alternative formulation via the Target Score Identity (TSI). However, these estimators face a fundamental variance trade-off: DSI exhibits high variance in low-noise regimes, whereas TSI suffers from high variance at high noise levels. In this work, we reconcile these approaches by unifying both estimators within the principled framework of control variates. We introduce the Control Variate Score Identity (CVSI), deriving an optimal, time-dependent control coefficient that theoretically guarantees variance minimization across the entire noise spectrum. We demonstrate that CVSI serves as a robust, low-variance plug-in estimator that significantly enhances sample efficiency in both data-free sampler learning and inference-time diffusion sampling.