Sampling in Unit Time with Kernel Fisher-Rao Flow

TL;DR

This paper presents a novel gradient-free sampling method using a mean-field ODE derived from Fisher-Rao flow, employing RKHS for tractability, and demonstrates its effectiveness through empirical results.

Contribution

It introduces a new gradient-free particle system based on Fisher-Rao flow and a tractable RKHS-based velocity field for efficient sampling from unnormalized densities.

Findings

Outperforms comparable gradient-free particle systems.

Produces high-quality samples from diverse target distributions.

Empirically competitive with gradient-based sampling methods.

Abstract

We introduce a new mean-field ODE and corresponding interacting particle systems (IPS) for sampling from an unnormalized target density. The IPS are gradient-free, available in closed form, and only require the ability to sample from a reference density and compute the (unnormalized) target-to-reference density ratio. The mean-field ODE is obtained by solving a Poisson equation for a velocity field that transports samples along the geometric mixture of the two densities, which is the path of a particular Fisher-Rao gradient flow. We employ a RKHS ansatz for the velocity field, which makes the Poisson equation tractable and enables discretization of the resulting mean-field ODE over finite samples. The mean-field ODE can be additionally be derived from a discrete-time perspective as the limit of successive linearizations of the Monge-Amp\`ere equations within a framework known as sample-driven optimal transport. We introduce a stochastic variant of our approach and demonstrate empirically that our IPS can produce high-quality samples from varied target distributions, outperforming comparable gradient-free particle systems and competitive with gradient-based alternatives.