Technical Note on Relating Scores of Tilted Distributions

TL;DR

This paper extends the understanding of how scores of tilted distributions relate to original distributions, especially in the context of score-based diffusion models, potentially improving score estimation techniques.

Contribution

It generalizes previous results to include negative diagonal tilts, linking tilted scores to original scores via location and noise level shifts.

Findings

Scores of tilted densities can be expressed in terms of original densities with shifted location and noise level.

The results apply to both linear and quadratic tilts, broadening the scope of score relationships.

Potential for improved score estimators in diffusion models based on these relationships.

Abstract

Recent results have shown that for a linear tilt to a reference measure, the scores that would be produced under convolution with a normal variable can be expressed in terms of convolutions of the original density. Here, we extend that result to include constant negative diagonal tilts as well. The relationship follows from relating the denoisers of the two densities, which define the scores via Tweedie formula. A linear tilt results in a location shift to the score operator, while a quadratic tilt results in both a location shift and a time shift. Thus the scores of the tilted density can be understood as the scores of the original convolution process at a different location and noise level. These results are of interest to those in the score based diffusion community, and may lead to better score estimators which take advantage of these tilted score relationships.