Propagation of Chaos for Mean-Field Langevin Dynamics and its Application to Model Ensemble

TL;DR

This paper improves the theoretical understanding of mean-field Langevin dynamics by refining propagation of chaos results, leading to better bounds and a new model ensemble method with guarantees.

Contribution

It refines the propagation of chaos analysis for MFLD by removing exponential dependence on regularization, and introduces a PoC-based ensemble strategy with theoretical assurances.

Findings

Improved propagation of chaos bounds for MFLD.

Elimination of exponential dependence on regularization coefficient.

Proposed model ensemble method with theoretical guarantees.

Abstract

Mean-field Langevin dynamics (MFLD) is an optimization method derived by taking the mean-field limit of noisy gradient descent for two-layer neural networks in the mean-field regime. Recently, the propagation of chaos (PoC) for MFLD has gained attention as it provides a quantitative characterization of the optimization complexity in terms of the number of particles and iterations. A remarkable progress by Chen et al. (2022) showed that the approximation error due to finite particles remains uniform in time and diminishes as the number of particles increases. In this paper, by refining the defective log-Sobolev inequality -- a key result from that earlier work -- under the neural network training setting, we establish an improved PoC result for MFLD, which removes the exponential dependence on the regularization coefficient from the particle approximation term of the optimization complexity. As an application, we propose a PoC-based model ensemble strategy with theoretical guarantees.