Improved Particle Approximation Error for Mean Field Neural Networks

TL;DR

This paper improves the theoretical understanding of particle approximation errors in mean-field neural networks by removing dependence on LSI constants, leading to better convergence and sampling guarantees.

Contribution

It establishes an LSI-constant-free particle approximation error for MFLD, enhancing the analysis of convergence and propagation of chaos in mean-field neural networks.

Findings

Improved convergence rates for MFLD.

Enhanced sampling guarantees for stationary distributions.

Uniform-in-time propagation of chaos with reduced particle complexity.

Abstract

Mean-field Langevin dynamics (MFLD) minimizes an entropy-regularized nonlinear convex functional defined over the space of probability distributions. MFLD has gained attention due to its connection with noisy gradient descent for mean-field two-layer neural networks. Unlike standard Langevin dynamics, the nonlinearity of the objective functional induces particle interactions, necessitating multiple particles to approximate the dynamics in a finite-particle setting. Recent works (Chen et al., 2022; Suzuki et al., 2023b) have demonstrated the uniform-in-time propagation of chaos for MFLD, showing that the gap between the particle system and its mean-field limit uniformly shrinks over time as the number of particles increases. In this work, we improve the dependence on logarithmic Sobolev inequality (LSI) constants in their particle approximation errors, which can exponentially deteriorate with the regularization coefficient. Specifically, we establish an LSI-constant-free particle approximation error concerning the objective gap by leveraging the problem structure in risk minimization. As the application, we demonstrate improved convergence of MFLD, sampling guarantee for the mean-field stationary distribution, and uniform-in-time Wasserstein propagation of chaos in terms of particle complexity.