Bidirectional Normalizing Flow: From Data to Noise and Back

TL;DR

This paper introduces Bidirectional Normalizing Flow (BiFlow), a flexible framework that improves generative modeling by learning an approximate inverse, leading to better quality and faster sampling than traditional NF methods.

Contribution

BiFlow removes the need for exact invertibility in normalizing flows, enabling more flexible architectures and loss functions for improved generative performance.

Findings

BiFlow outperforms causal decoding NF models in image generation quality.

BiFlow accelerates sampling by up to two orders of magnitude.

BiFlow achieves state-of-the-art results among NF-based methods.

Abstract

Normalizing Flows (NFs) have been established as a principled framework for generative modeling. Standard NFs consist of a forward process and a reverse process: the forward process maps data to noise, while the reverse process generates samples by inverting it. Typical NF forward transformations are constrained by explicit invertibility, ensuring that the reverse process can serve as their exact analytic inverse. Recent developments in TARFlow and its variants have revitalized NF methods by combining Transformers and autoregressive flows, but have also exposed causal decoding as a major bottleneck. In this work, we introduce Bidirectional Normalizing Flow ($\textbf{BiFlow}$), a framework that removes the need for an exact analytic inverse. BiFlow learns a reverse model that approximates the underlying noise-to-data inverse mapping, enabling more flexible loss functions and architectures. Experiments on ImageNet demonstrate that BiFlow, compared to its causal decoding counterpart, improves generation quality while accelerating sampling by up to two orders of magnitude. BiFlow yields state-of-the-art results among NF-based methods and competitive performance among single-evaluation ("1-NFE") methods. Following recent encouraging progress on NFs, we hope our work will draw further attention to this classical paradigm.