A Unified View of Score-Based and Drifting Models

TL;DR

This paper establishes a theoretical connection between drifting models and score-based generative models, showing that drifting approximates score-matching on smoothed distributions, especially with Laplace kernels.

Contribution

It provides a precise mathematical link between drifting and score-based models, revealing that drifting approximates score-matching through kernel estimates, bridging two prominent generative modeling approaches.

Findings

Gaussian kernels lead to an exact score-matching interpretation.

For Laplace kernels, the residual term is negligible in high dimensions.

Drifting and diffusion models both use score-mismatch transport directions.

Abstract

Drifting models train one-step generators by optimizing a kernel-induced mean-shift discrepancy between the data and model distributions, with Laplace kernels used by default in practice. At each point, this discrepancy compares the kernel-weighted displacement toward nearby data samples with the corresponding displacement toward nearby model samples, thereby defining a transport direction for generated samples. In this paper, we show that drifting is more closely connected to score-based generative modeling than it may first appear, establishing a precise link to the score-matching principle underlying diffusion models. For Gaussian kernels, the population mean-shift field exactly equals the difference between the scores (i.e., the gradient-log-densities) of the Gaussian-smoothed data and model distributions. This identity follows from Tweedie's formula, which links the score of a Gaussian-smoothed density to its conditional mean, and implies that Gaussian-kernel drifting is exactly a score-matching objective on smoothed distributions. More generally, we derive an exact decomposition for radial kernels in which mean shift equals a score-based field plus a residual term. For the practical Laplace kernel, we further show theoretically and empirically that this residual is negligible in high dimension, implying that the transport field used in practice is nearly score-based. Our results reveal a structural connection to diffusion models: both methods use score-mismatch transport directions, but drifting realizes the score nonparametrically through kernel-based estimates, whereas diffusion models learn it parametrically with neural networks.