Binary interaction methods for high dimensional global optimization and machine learning

TL;DR

This paper introduces a new gradient-free global optimization method based on binary interaction dynamics, extending consensus-based optimization, with applications demonstrated in machine learning.

Contribution

It presents a novel class of binary interaction-based optimization algorithms derived from Boltzmann dynamics, generalizing existing consensus methods.

Findings

Convergence to global minimizers shown for a broad class of functions.

Algorithmic implementation using Monte Carlo methods.

Successful application to prototype test functions and machine learning tasks.

Abstract

In this work we introduce a new class of gradient-free global optimization methods based on a binary interaction dynamics governed by a Boltzmann type equation. In each interaction the particles act taking into account both the best microscopic binary position and the best macroscopic collective position. In the mean-field limit we show that the resulting Fokker-Planck partial differential equations generalize the current class of consensus based optimization (CBO) methods. For the latter methods, convergence to the global minimizer can be shown for a large class of functions. Algorithmic implementations inspired by the well-known direct simulation Monte Carlo methods in kinetic theory are derived and discussed. Several examples on prototype test functions for global optimization are reported including applications to machine learning.