Diffusion Models with Heavy-Tailed Targets: Score Estimation and Sampling Guarantees

TL;DR

This paper analyzes score-based diffusion models for heavy-tailed data, deriving minimax rates for score estimation and sampling guarantees, and highlighting differences between exponential and polynomial tail decay.

Contribution

It extends diffusion model theory to heavy-tailed distributions, providing sharp minimax rates and sampling guarantees under broad tail conditions.

Findings

Sharp minimax rates for score estimation under heavy tails

Sampling guarantees with convergence rates depending on tail decay

Extension of diffusion models beyond light-tailed assumptions

Abstract

Score-based diffusion models have become a powerful framework for generative modeling, with score estimation as a central statistical bottleneck. Existing guarantees for score estimation largely focus on light-tailed targets or rely on restrictive assumptions such as compact support, which are often violated by heavy-tailed data in practice. In this work, we study conventional (Gaussian) score-based diffusion models when the target distribution is heavy-tailed and belongs to a Sobolev class with smoothness parameter $\beta>0$. We consider both exponential and polynomial tail decay, indexed by a tail parameter $\gamma$. Using kernel density estimation, we derive sharp minimax rates for score estimation, revealing a qualitative dichotomy: under exponential tails, the rate matches the light-tailed case up to polylogarithmic factors, whereas under polynomial tails the rate depends explicitly on $\gamma$. We further provide sampling guarantees for the associated continuous reverse dynamics. In total variation, the generated distribution converges at the minimax optimal rate $n^{-\beta/(2\beta+d)}$ under exponential tails (up to logarithmic factors), and at a $\gamma$-dependent rate under polynomial tails. Whether the latter sampling rate is minimax optimal remains an open question. These results characterize the statistical limits of score estimation and the resulting sampling accuracy for heavy-tailed targets, extending diffusion theory beyond the light-tailed setting.