Semi-Autoregressive Energy Flows: Exploring Likelihood-Free Training of Normalizing Flows

TL;DR

This paper introduces likelihood-free training for normalizing flows using an energy-based objective, enabling flexible architectures like semi-autoregressive energy flows, which achieve competitive results without relying on likelihood estimates.

Contribution

It proposes the energy objective for training flows without likelihood, and introduces semi-autoregressive energy flows as a new model family.

Findings

Energy flows have competitive sample quality.

Likelihood-free training decouples performance from likelihood estimates.

Semi-autoregressive energy flows interpolate between autoregressive and non-autoregressive models.

Abstract

Training normalizing flow generative models can be challenging due to the need to calculate computationally expensive determinants of Jacobians. This paper studies the likelihood-free training of flows and proposes the energy objective, an alternative sample-based loss based on proper scoring rules. The energy objective is determinant-free and supports flexible model architectures that are not easily compatible with maximum likelihood training, including semi-autoregressive energy flows, a novel model family that interpolates between fully autoregressive and non-autoregressive models. Energy flows feature competitive sample quality, posterior inference, and generation speed relative to likelihood-based flows; this performance is decorrelated from the quality of log-likelihood estimates, which are generally very poor. Our findings question the use of maximum likelihood as an objective or a metric, and contribute to a scientific study of its role in generative modeling.