Energy-Tweedie: Score meets Score, Energy meets Energy

TL;DR

This paper extends Tweedie's formula to elliptical energy models, linking denoising, score estimation, and energy scores, enabling broader applications in diffusion models and noise distribution analysis.

Contribution

It generalizes Tweedie's formula to elliptical energy models and introduces an identity connecting energy scores to noisy marginals, broadening diffusion model applications.

Findings

Extended Tweedie's formula to elliptical distributions.

Derived a fundamental identity linking energy scores and noisy marginals.

Enabled new methods for score and noise parameter estimation.

Abstract

Denoising and score estimation have long been known to be linked via the classical Tweedie's formula. In this work, we first extend the latter to a wider range of distributions often called "energy models" and denoted elliptical distributions in this work. Next, we examine an alternative view: we consider the denoising posterior $P(X|Y)$ as the optimizer of the energy score (a scoring rule) and derive a fundamental identity that connects the (path-) derivative of a (possibly) non-Euclidean energy score to the score of the noisy marginal. This identity can be seen as an analog of Tweedie's identity for the energy score, and allows for several interesting applications; for example, score estimation, noise distribution parameter estimation, as well as using energy score models in the context of "traditional" diffusion model samplers with a wider array of noising distributions.