Generalizing Score-based generative models for Heavy-tailed Distributions

TL;DR

This paper extends score-based generative models to heavy-tailed distributions through theoretical analysis and proposes a unified framework combining normalizing flows and SGMs for improved generative fidelity.

Contribution

It provides the first theoretical guarantees for extending SGMs to heavy-tailed data and introduces a unified approach combining normalizing flows with SGMs.

Findings

Proves convergence of the diffusion process with early stopping and proper initialization.

Establishes theoretical guarantees for normalizing flows on heavy-tailed distributions.

Demonstrates improved generative quality by combining flows and SGMs.

Abstract

Score-based generative models (SGMs) have achieved remarkable empirical success, motivating their application to a broad range of data distributions. However, extending them to heavy-tailed targets remains a largely open problem. Although dedicated models for heavy-tailed distributions have been proposed, their generative fidelity remains unclear and they lack solid theoretical foundations, leaving important questions open in this regime. In this paper, we address this gap through two theoretical contributions. First, we show that combining early stopping with a suitable initialization is sufficient to extend the diffusion framework to any target distribution; in particular, we establish the well-posedness of the backward process and prove convergence of the approximated diffusion in KL divergence. Second, we derive novel theoretical guarantees for generation with normalizing flows, obtaining convergence results that hold under mild conditions on the flow family and without any assumption on the tail behavior of the target distribution. Building on these results, we propose a unified generative framework for heavy-tailed distributions: a normalizing flow is first trained to capture the tail behavior and is then used as an initialization prior for an SGM, which refines the samples by recovering fine-grained structural details. This design leverages the complementary strengths of the two model classes within a theoretically principled pipeline, overcoming the limitations of existing approaches.