A multiscale Consensus-Based algorithm for multi-level optimization

TL;DR

This paper introduces a multiscale consensus-based optimization algorithm that employs multiple interacting particle populations to efficiently solve complex bi- and tri-level optimization problems, with proven convergence and demonstrated effectiveness.

Contribution

The paper generalizes existing CBO techniques by incorporating multiscale dynamics and multiple populations for multi-level optimization, providing theoretical convergence analysis and practical numerical validation.

Findings

Effective in solving complex multi-level optimization tasks

Performs well on min-max and saddle point problems

Converges to an averaged effective dynamics as time-scale separation approaches zero

Abstract

A novel multiscale consensus-based optimization (CBO) algorithm for solving bi- and tri-level optimization problems is introduced. Existing CBO techniques are generalized by the proposed method through the employment of multiple interacting populations of particles, each of which is used to optimize one level of the problem. These particle populations are evolved through multiscale-in-time dynamics, which are formulated as a singularly perturbed system of stochastic differential equations. Theoretical convergence analysis for the multiscale CBO model to an averaged effective dynamics as the time-scale separation parameter approaches zero is provided. The resulting algorithm is presented for both bi-level and tri-level optimization problems. The effectiveness of the approach in tackling complex multi-level optimization tasks is demonstrated through numerical experiments on various benchmark functions. Additionally, it is shown that the proposed method performs well on min-max optimization problems, comparing favorably with existing CBO algorithms for saddle point problems.