Swarm-based optimization with jumps: a kinetic BGK framework and convergence analysis

TL;DR

This paper introduces a kinetic BGK framework for swarm-based optimization with jumps, providing convergence analysis and connecting it to existing methods like CBO, enhancing stochastic exploration in non-convex optimization.

Contribution

It formalizes a novel particle-based optimization algorithm with jump velocities using a kinetic BGK model, unifying various noise distributions and analyzing convergence.

Findings

Proves propagation of chaos and convergence for Gaussian noise in bounded domains.

Demonstrates the method's effectiveness on benchmark problems.

Shows the connection between the proposed model and CBO dynamics.

Abstract

Metaheuristic algorithms are powerful tools for global optimization, particularly for non-convex and non-differentiable problems where exact methods are often impractical. Particle-based optimization methods, inspired by swarm intelligence principles, have shown effectiveness due to their ability to balance exploration and exploitation within the search space. In this work, we introduce a novel particle-based optimization algorithm where velocities are updated via random jumps, a strategy commonly used to enhance stochastic exploration. We formalize this approach by describing the dynamics through a kinetic modelling of BGK type, offering a unified framework that accommodates general noise distributions, including heavy-tailed ones like Cauchy. Under suitable parameter scaling, the model reduces to the Consensus-Based Optimization (CBO) dynamics. For non-degenerate Gaussian noise in bounded domains, we prove propagation of chaos and convergence towards minimizers. Numerical results on benchmark problems validate the approach and highlight its connection to CBO.