Strong Global Convergence of the Consensus-Based Optimization Algorithm

TL;DR

This paper proves strong mean square convergence of the consensus-based optimization algorithm, a derivative-free method for global minimization, providing explicit rates and conditions for both isotropic and anisotropic diffusion cases.

Contribution

It establishes the first rigorous proof of strong convergence for the practical time-discrete CBO algorithm, including explicit rates and conditions on hyperparameters.

Findings

Proves strong mean square convergence to the global minimizer.

Provides explicit convergence rates in time step size and number of particles.

Extends analysis to both isotropic and anisotropic diffusion cases.

Abstract

Consensus-based optimization (CBO) is a multi-agent metaheuristic derivative-free optimization algorithm that has proven to be capable of globally minimizing nonconvex nonsmooth functions across a diverse range of applications while being amenable to theoretical analysis. The method leverages an interplay between exploration of the energy landscape of the objective function through a system of interacting particles subject to stochasticity and exploitation of the particles' positions through the computation of a global consensus about the location of the minimizer based on the Laplace principle. In this paper, we prove strong mean square convergence of the practical numerical time-discrete CBO algorithm to the global minimizer for a rich class of objective functions. For CBO with both isotropic and anisotropic diffusion, our convergence result features conditions on the choice of the hyperparameters as well as explicit rates of convergence in the time discretization step size $\Delta t$ and the number of particles $N$. By interpreting the time-discrete algorithm at the continuous-time level through a system of stochastic differential equations (SDEs), our proof strategy combines traditional finite-time convergence theory for numerical methods applied to SDEs with careful considerations due to the fact that the CBO coefficients do not satisfy a global Lipschitz condition. To accomodate the latter, we adopt a recently proposed generalization of Sznitman's classical argument, which allows to discard an event of small probability, controllable through fine moment estimates for the particle systems.