Analytic Bijections for Smooth and Interpretable Normalizing Flows

TL;DR

This paper introduces three new globally smooth, analytically invertible bijections for normalizing flows, improving expressivity, stability, and interpretability, and demonstrates their effectiveness on various benchmarks and physics problems.

Contribution

The paper presents three novel families of analytic bijections—cubic rational, sinh, and cubic polynomial—that are smooth, invertible, and applicable as drop-in replacements in normalizing flows, enhancing their flexibility and performance.

Findings

New bijections outperform spline-based flows in benchmarks.

Radial flows offer stable training and interpretability.

Method scales effectively to high-dimensional physics problems.

Abstract

A key challenge in designing normalizing flows is finding expressive scalar bijections that remain invertible with tractable Jacobians. Existing approaches face trade-offs: affine transformations are smooth and analytically invertible but lack expressivity; monotonic splines offer local control but are only piecewise smooth and act on bounded domains; residual flows achieve smoothness but need numerical inversion. We introduce three families of analytic bijections -- cubic rational, sinh, and cubic polynomial -- that are globally smooth ($C^\infty$), defined on all of $\mathbb{R}$, and analytically invertible in closed form, combining the favorable properties of all prior approaches. These bijections serve as drop-in replacements in coupling flows, matching or exceeding spline performance. Beyond coupling layers, we develop radial flows: a novel architecture using direct parametrization that transforms the radial coordinate while preserving angular direction. Radial flows exhibit exceptional training stability, produce geometrically interpretable transformations, and on targets with radial structure can achieve comparable quality to coupling flows with $1000\times$ fewer parameters. We provide comprehensive evaluation on 1D and 2D benchmarks, and demonstrate applicability to higher-dimensional physics problems through experiments on $\phi^4$ lattice field theory, where our bijections outperform affine baselines and enable problem-specific designs that address mode collapse.