From particle swarm optimization to consensus based optimization: stochastic modeling and mean-field limit

TL;DR

This paper develops a stochastic differential equation model for particle swarm optimization, derives its mean-field limit, and connects it to consensus-based optimization, providing insights and numerical validation for these global optimization techniques.

Contribution

It introduces a mean-field framework for PSO, overcoming memory effects, and links it to CBO through hydrodynamic limits, advancing the theoretical understanding of these methods.

Findings

Mean-field equations accurately describe PSO dynamics.

Small inertia limit leads to hydrodynamic equations related to CBO.

Numerical examples demonstrate the effectiveness of the mean-field approach.

Abstract

In this paper we consider a continuous description based on stochastic differential equations of the popular particle swarm optimization (PSO) process for solving global optimization problems and derive in the large particle limit the corresponding mean-field approximation based on Vlasov-Fokker-Planck-type equations. The disadvantage of memory effects induced by the need to store the local best position is overcome by the introduction of an additional differential equation describing the evolution of the local best. A regularization process for the global best permits to formally derive the respective mean-field description. Subsequently, in the small inertia limit, we compute the related macroscopic hydrodynamic equations that clarify the link with the recently introduced consensus based optimization (CBO) methods. Several numerical examples illustrate the mean field process, the small inertia limit and the potential of this general class of global optimization methods.