Flexible Tails for Normalizing Flows

TL;DR

This paper introduces Tail Transform Flows (TTF), a novel method that enables normalizing flows to model heavy-tailed distributions effectively using a Gaussian base and a specialized tail transformation.

Contribution

The paper proposes TTF, a new approach that improves modeling of heavy-tailed distributions without heavy-tailed base distributions, enhancing optimization and performance.

Findings

TTF outperforms existing methods in high-dimensional settings.

TTF effectively models heavy tails with better optimization stability.

Experimental results demonstrate superior accuracy in tail-heavy distribution tasks.

Abstract

Normalizing flows are a flexible class of probability distributions, expressed as transformations of a simple base distribution. A limitation of standard normalizing flows is representing distributions with heavy tails, which arise in applications to both density estimation and variational inference. A popular current solution to this problem is to use a heavy tailed base distribution. We argue this can lead to poor performance due to the difficulty of optimising neural networks, such as normalizing flows, under heavy tailed input. We propose an alternative, "tail transform flow" (TTF), which uses a Gaussian base distribution and a final transformation layer which can produce heavy tails. Experimental results show this approach outperforms current methods, especially when the target distribution has large dimension or tail weight.