Kernel Approximation of Fisher-Rao Gradient Flows

TL;DR

This paper provides a rigorous theoretical framework for kernel approximations of Fisher-Rao gradient flows, connecting PDE gradient flows, optimal transport, and machine learning applications, with convergence guarantees and energy dissipation analysis.

Contribution

It introduces a unified theoretical foundation for kernel-based approximations of Fisher-Rao flows, including convergence analysis and connections to learning algorithms.

Findings

Proves evolutionary Γ-convergence for kernel-approximated Fisher-Rao flows.

Characterizes gradient flows in the maximum-mean discrepancy space.

Establishes links between Fisher-Rao flows, Stein flows, and kernel discrepancies.

Abstract

The purpose of this paper is to answer a few open questions in the interface of kernel methods and PDE gradient flows. Motivated by recent advances in machine learning, particularly in generative modeling and sampling, we present a rigorous investigation of Fisher-Rao and Wasserstein type gradient flows concerning their gradient structures, flow equations, and their kernel approximations. Specifically, we focus on the Fisher-Rao (also known as Hellinger) geometry and its various kernel-based approximations, developing a principled theoretical framework using tools from PDE gradient flows and optimal transport theory. We also provide a complete characterization of gradient flows in the maximum-mean discrepancy (MMD) space, with connections to existing learning and inference algorithms. Our analysis reveals precise theoretical insights linking Fisher-Rao flows, Stein flows, kernel discrepancies, and nonparametric regression. We then rigorously prove evolutionary $\Gamma$-convergence for kernel-approximated Fisher-Rao flows, providing theoretical guarantees beyond pointwise convergence. Finally, we analyze energy dissipation using the Helmholtz-Rayleigh principle, establishing important connections between classical theory in mechanics and modern machine learning practice. Our results provide a unified theoretical foundation for understanding and analyzing approximations of gradient flows in machine learning applications through a rigorous gradient flow and variational method perspective.