Training Energy-Based Normalizing Flow with Score-Matching Objectives

TL;DR

This paper introduces energy-based normalizing flows trained with score-matching, enabling efficient, stable training with arbitrary linear layers and achieving superior performance and speed compared to traditional maximum likelihood methods.

Contribution

The paper proposes a novel EBFlow model trained via score-matching, bypassing Jacobian determinant calculations and improving training efficiency and stability.

Findings

Achieves significant speedup over maximum likelihood training.

Outperforms prior methods in negative log-likelihood.

Enables use of arbitrary linear layers without increased complexity.

Abstract

In this paper, we establish a connection between the parameterization of flow-based and energy-based generative models, and present a new flow-based modeling approach called energy-based normalizing flow (EBFlow). We demonstrate that by optimizing EBFlow with score-matching objectives, the computation of Jacobian determinants for linear transformations can be entirely bypassed. This feature enables the use of arbitrary linear layers in the construction of flow-based models without increasing the computational time complexity of each training iteration from $O(D^2L)$ to $O(D^3L)$ for an $L$-layered model that accepts $D$-dimensional inputs. This makes the training of EBFlow more efficient than the commonly-adopted maximum likelihood training method. In addition to the reduction in runtime, we enhance the training stability and empirical performance of EBFlow through a number of techniques developed based on our analysis of the score-matching methods. The experimental results demonstrate that our approach achieves a significant speedup compared to maximum likelihood estimation while outperforming prior methods with a noticeable margin in terms of negative log-likelihood (NLL).