Learning Interpolations between Boltzmann Densities

TL;DR

This paper proposes a novel training objective for continuous normalizing flows that interpolates between energy functions to model complex distributions without requiring samples, demonstrated on Gaussian mixtures and quantum densities.

Contribution

Introduces an energy interpolation-based training method for continuous normalizing flows that does not depend on sample data, enabling modeling of complex distributions.

Findings

Effective on Gaussian mixtures

Successfully models quantum Boltzmann densities

Outperforms reverse KL-divergence in experiments

Abstract

We introduce a training objective for continuous normalizing flows that can be used in the absence of samples but in the presence of an energy function. Our method relies on either a prescribed or a learnt interpolation $f_t$ of energy functions between the target energy $f_1$ and the energy function of a generalized Gaussian $f_0(x) = ||x/\sigma||_p^p$. The interpolation of energy functions induces an interpolation of Boltzmann densities $p_t \propto e^{-f_t}$ and we aim to find a time-dependent vector field $V_t$ that transports samples along the family $p_t$ of densities. The condition of transporting samples along the family $p_t$ is equivalent to satisfying the continuity equation with $V_t$ and $p_t = Z_t^{-1}e^{-f_t}$. Consequently, we optimize $V_t$ and $f_t$ to satisfy this partial differential equation. We experimentally compare the proposed training objective to the reverse KL-divergence on Gaussian mixtures and on the Boltzmann density of a quantum mechanical particle in a double-well potential.