Verlet Flows: Exact-Likelihood Integrators for Flow-Based Generative Models

TL;DR

Verlet flows are a new class of continuous normalizing flows that use symplectic integrators to enable exact likelihood computation, improving importance sampling efficiency in generative modeling.

Contribution

Introduction of Verlet flows, which leverage Hamiltonian-inspired symplectic integrators for exact likelihoods in flow-based models, extending non-continuous architectures with minimal expressivity constraints.

Findings

Verlet flows match autograd trace accuracy while being faster.

Variance of Hutchinson trace estimator is unsuitable for importance sampling.

Verlet flows perform well on toy density experiments.

Abstract

Approximations in computing model likelihoods with continuous normalizing flows (CNFs) hinder the use of these models for importance sampling of Boltzmann distributions, where exact likelihoods are required. In this work, we present Verlet flows, a class of CNFs on an augmented state-space inspired by symplectic integrators from Hamiltonian dynamics. When used with carefully constructed Taylor-Verlet integrators, Verlet flows provide exact-likelihood generative models which generalize coupled flow architectures from a non-continuous setting while imposing minimal expressivity constraints. On experiments over toy densities, we demonstrate that the variance of the commonly used Hutchinson trace estimator is unsuitable for importance sampling, whereas Verlet flows perform comparably to full autograd trace computations while being significantly faster.