Bayesian inference via sparse Hamiltonian flows

TL;DR

This paper introduces sparse Hamiltonian flows, a novel method for constructing Bayesian coresets efficiently, achieving exponential data compression and accurate posterior inference with reduced computational cost.

Contribution

The paper presents a new approach combining Hamiltonian flows and momentum quasi-refreshment to efficiently build Bayesian coresets with theoretical guarantees and practical improvements.

Findings

Achieves exponential data compression in representative models.

Reduces KL divergence to the target through momentum quasi-refreshment.

Demonstrates significantly faster runtime compared to existing methods.

Abstract

A Bayesian coreset is a small, weighted subset of data that replaces the full dataset during Bayesian inference, with the goal of reducing computational cost. Although past work has shown empirically that there often exists a coreset with low inferential error, efficiently constructing such a coreset remains a challenge. Current methods tend to be slow, require a secondary inference step after coreset construction, and do not provide bounds on the data marginal evidence. In this work, we introduce a new method -- sparse Hamiltonian flows -- that addresses all three of these challenges. The method involves first subsampling the data uniformly, and then optimizing a Hamiltonian flow parametrized by coreset weights and including periodic momentum quasi-refreshment steps. Theoretical results show that the method enables an exponential compression of the dataset in a representative model, and that the quasi-refreshment steps reduce the KL divergence to the target. Real and synthetic experiments demonstrate that sparse Hamiltonian flows provide accurate posterior approximations with significantly reduced runtime compared with competing dynamical-system-based inference methods.