Tweedie Calculus

TL;DR

This paper develops a general theoretical framework for Tweedie representations in additive-noise models, extending classical formulas to non-Gaussian noise and complex functionals, with applications in privacy and heteroskedastic models.

Contribution

It introduces the Tweedie functional, a linear map characterizing when posterior expectations admit direct density-based formulas, generalizing classical results.

Findings

Classical Gaussian Tweedie formula is recovered as a special case.

New Tweedie representations are derived for non-Gaussian noise models.

Application to differential privacy mechanisms demonstrates practical utility.

Abstract

Tweedie's formula is central to measurement-error analysis and empirical Bayes. Under Gaussian noise, the formula identifies the posterior mean directly from the observed-data density, bypassing nonparametric deconvolution. Beyond a few classical examples, however, no general theory explains when analogous identities hold, how they are structured, or how to derive them for non-Gaussian noise and for posterior functionals other than the mean. This paper develops such a framework for additive-noise models. I characterize when conditional expectations of an unobserved latent variable, given the observed signal, admit direct expressions in terms of the observed density -- identities I call Tweedie representations -- and show that they are governed by a linear map, the Tweedie functional. Under general conditions, I prove that this functional exists, is unique, and is continuous. I also provide a constructive method for deriving it by extending the inverse Fourier transform of an explicit tempered distribution. This recasts the search for Tweedie-type formulas as a problem in the calculus of tempered distributions. The framework recovers the classical Gaussian formula and yields new representations for posterior means under non-Gaussian noise. I apply the method to construct unbiased representations of nonlinear functionals of latent variables and to derive Tweedie formulas for the product-Laplace mechanism used in differential privacy. Finally, I show that the approach extends beyond the standard additive model. In the heteroskedastic Gaussian sequence model, where the noise covariance is itself random, a change of variables restores the required additive-noise structure conditionally, yielding Tweedie representations without additional restrictions on the joint law of the latent parameter and noise covariance.