Sequential Monte Carlo approximations of Wasserstein--Fisher--Rao gradient flows

TL;DR

This paper introduces a new Monte Carlo algorithm based on Wasserstein--Fisher--Rao geometry to efficiently sample from complex probability distributions, demonstrating its advantages over existing methods through extensive experiments.

Contribution

It develops a novel sequential Monte Carlo algorithm for Wasserstein--Fisher--Rao gradient flows targeting KL divergence minimization.

Findings

The proposed algorithm effectively approximates Wasserstein--Fisher--Rao flows.

It outperforms traditional Monte Carlo methods in specific sampling scenarios.

Empirical results highlight the conditions where the new method is most advantageous.

Abstract

We consider the problem of sampling from a probability distribution $\pi$. It is well known that this can be written as an optimisation problem over the space of probability distribution in which we aim to minimise the Kullback--Leibler divergence from $\pi$. We consider several partial differential equations (PDEs) whose solution is a minimiser of the Kullback--Leibler divergence from $\pi$ and connect them to well-known Monte Carlo algorithms. We focus in particular on PDEs obtained by considering the Wasserstein--Fisher--Rao geometry over the space of probabilities and show that these lead to a natural implementation using importance sampling and sequential Monte Carlo. We propose a novel algorithm to approximate the Wasserstein--Fisher--Rao flow of the Kullback--Leibler divergence and conduct an extensive empirical study to identify when these algorithms outperforms other popular Monte Carlo algorithms.