Trends in Consensus-based optimization

TL;DR

This paper reviews consensus-based optimization algorithms, their variants, analytical properties, and applications, highlighting their suitability for high-dimensional and constrained problems in machine learning and engineering.

Contribution

It provides a comprehensive overview of consensus-based optimization, including recent variants, analytical results, and connections to other methods like particle swarm optimization.

Findings

Analytical estimates are dimension independent.

Variants using component-wise or common noise are discussed.

Applications include high-dimensional and constrained optimization problems.

Abstract

In this chapter we give an overview of the consensus-based global optimization algorithm and its recent variants. We recall the formulation and analytical results of the original model, then we discuss variants using component-wise independent or common noise. In combination with mini-batch approaches those variants were tailored for machine learning applications. Moreover, it turns out that the analytical estimates are dimension independent, which is useful for high-dimensional problems. We discuss the relationship of consensus-based optimization with particle swarm optimization, a method widely used in the engineering community. Then we survey a variant of consensus-based optimization that is proposed for global optimization problems constrained to hyper-surfaces. We conclude the chapter with remarks on applications, preprints and open problems.