SCORENF: Score-based Normalizing Flows for Sampling Unnormalized distributions

TL;DR

ScoreNF introduces a score-based normalizing flow framework combined with an MCMC module to efficiently sample from unnormalized distributions, outperforming traditional methods especially with limited training data.

Contribution

The paper presents ScoreNF, a novel score-based normalizing flow method with an integrated MCMC component for unbiased sampling from unnormalized distributions, reducing dependence on large datasets.

Findings

Effective sampling of synthetic 2D distributions and high-dimensional lattice field theory.

Maintains high performance with small training ensembles.

Provides a method to assess mode-covering and mode-collapse behaviors.

Abstract

Unnormalized probability distributions are central to modeling complex physical systems across various scientific domains. Traditional sampling methods, such as Markov Chain Monte Carlo (MCMC), often suffer from slow convergence, critical slowing down, poor mode mixing, and high autocorrelation. In contrast, likelihood-based and adversarial machine learning models, though effective, are heavily data-driven, requiring large datasets and often encountering mode covering and mode collapse. In this work, we propose ScoreNF, a score-based learning framework built on the Normalizing Flow (NF) architecture, integrated with an Independent Metropolis-Hastings (IMH) module, enabling efficient and unbiased sampling from unnormalized target distributions. We show that ScoreNF maintains high performance even with small training ensembles, thereby reducing reliance on computationally expensive MCMC-generated training data. We also present a method for assessing mode-covering and mode-collapse behaviours. We validate our method on synthetic 2D distributions (MOG-4 and MOG-8) and the high-dimensional $\phi^4$ lattice field theory distribution, demonstrating its effectiveness for sampling tasks.