Training Normalizing Flows from Dependent Data

TL;DR

This paper introduces a new likelihood-based training method for normalizing flows that accounts for dependencies between data points, improving density estimation and data generation in dependent data scenarios.

Contribution

It proposes a novel likelihood objective and learning algorithm for normalizing flows that explicitly models dependencies among data points, addressing a key limitation of existing methods.

Findings

Improved empirical results on synthetic and real-world data.

Enhanced statistical power in genome-wide association studies.

Better density estimation when data dependencies are modeled.

Abstract

Normalizing flows are powerful non-parametric statistical models that function as a hybrid between density estimators and generative models. Current learning algorithms for normalizing flows assume that data points are sampled independently, an assumption that is frequently violated in practice, which may lead to erroneous density estimation and data generation. We propose a likelihood objective of normalizing flows incorporating dependencies between the data points, for which we derive a flexible and efficient learning algorithm suitable for different dependency structures. We show that respecting dependencies between observations can improve empirical results on both synthetic and real-world data, and leads to higher statistical power in a downstream application to genome-wide association studies.