Phase-Type Variational Autoencoders for Heavy-Tailed Data

TL;DR

The paper introduces PH-VAE, a novel deep generative model with a flexible, analytically tractable decoder based on Phase-Type distributions, effectively capturing heavy-tailed behaviors and dependencies in complex data.

Contribution

It is the first to incorporate Phase-Type distributions into deep generative models, enabling adaptive tail modeling directly from data.

Findings

PH-VAE accurately models diverse heavy-tailed distributions.

It outperforms Gaussian, Student-t, and extreme-value VAEs in tail behavior.

Captures realistic cross-dimensional tail dependence in multivariate data.

Abstract

Heavy-tailed distributions are ubiquitous in real-world data, where rare but extreme events dominate risk and variability. However, standard Variational Autoencoders (VAEs) employ simple decoder distributions (e.g., Gaussian) that fail to capture heavy-tailed behavior, while existing heavy-tail-aware extensions remain restricted to predefined parametric families whose tail behavior is fixed a priori. We propose the Phase-Type Variational Autoencoder (PH-VAE), whose decoder distribution is a latent-conditioned Phase-Type (PH) distribution defined as the absorption time of a continuous-time Markov chain (CTMC). This formulation composes multiple exponential time scales, yielding a flexible and analytically tractable decoder that adapts its tail behavior directly from the observed data. Experiments on synthetic and real-world benchmarks demonstrate that PH-VAE accurately recovers diverse heavy-tailed distributions, significantly outperforming Gaussian, Student-t, and extreme-value-based VAE decoders in modeling tail behavior and extreme quantiles. In multivariate settings, PH-VAE captures realistic cross-dimensional tail dependence through its shared latent representation. To our knowledge, this is the first work to integrate Phase-Type distributions into deep generative modeling, bridging applied probability and representation learning.