Kinetic models for optimization: a unified mathematical framework for metaheuristics

TL;DR

This paper reviews how kinetic theory can unify and deepen the mathematical understanding of various metaheuristic optimization algorithms, potentially leading to new methods.

Contribution

It introduces a kinetic modeling framework that describes diverse metaheuristics under a common mathematical perspective, facilitating theoretical analysis and algorithm development.

Findings

Kinetic models provide a unified description of metaheuristics.

The framework enables derivation of new algorithms using alternative numerical methods.

Connections between different algorithms are clarified through asymptotic scalings.

Abstract

Metaheuristic algorithms, widely used for solving complex non-convex and non-differentiable optimization problems, often lack a solid mathematical foundation. In this review, we explore how concepts and methods from kinetic theory can offer a potential unifying framework for a variety of metaheuristic optimization methods. By applying principles from collisional and non-collisional kinetic theory, we outline how particle-based algorithms like Simulated Annealing, Genetic Algorithms, Particle Swarm Optimization, and Ensemble Kalman Filter may be described through a common statistical perspective. This approach not only provides a path to deeper theoretical insights and connects different methods under suitable asymptotic scalings, but also enables the derivation of novel algorithms using alternative numerical solvers. While not exhaustive, our review highlights how kinetic models can enhance the mathematical understanding of existing optimization algorithms and inspire new computational strategies.