On propagation of chaos for the Fisher-Rao gradient flow in entropic mean-field optimization

TL;DR

This paper studies the Fisher-Rao gradient flow in entropic mean-field optimization, establishing a rigorous link between the flow and particle systems, and proving propagation of chaos for the approximation algorithms.

Contribution

It introduces a kernelized approach to the Fisher-Rao gradient flow, proving existence, uniqueness, and propagation of chaos for the particle system approximation.

Findings

Proved existence and uniqueness of kernelized Fisher-Rao flows.

Established propagation of chaos for the particle approximation.

Provided theoretical justification for particle-based algorithms in mean-field optimization.

Abstract

We consider a class of optimization problems on the space of probability measures motivated by the mean-field approach to studying neural networks. Such problems can be solved by constructing continuous-time gradient flows that converge to the minimizer of the energy function under consideration, and then implementing discrete-time algorithms that approximate the flow. In this work, we focus on the Fisher-Rao gradient flow and we construct an interacting particle system that approximates the flow as its mean-field limit. We discuss the connection between the energy function, the gradient flow and the particle system and explain different approaches to smoothing out the energy function with an appropriate kernel in a way that allows for the particle system to be well-defined. We provide a rigorous proof of the existence and uniqueness of thus obtained kernelized flows, as well as a propagation of chaos result that provides a theoretical justification for using the corresponding kernelized particle systems as approximation algorithms in entropic mean-field optimization.