Constrained Consensus-Based Optimization and Numerical Heuristics for the Few Particle Regime

TL;DR

This paper extends consensus-based optimization (CBO) to constrained domains with boundary conditions, proves global convergence in the many-particle regime, and introduces heuristics and adaptive mechanisms to enhance performance in the few-particle regime, demonstrated on complex variational problems.

Contribution

It provides the first global convergence proof for constrained CBO with boundary conditions and develops heuristics for effective optimization with few particles.

Findings

Proved global convergence of constrained CBO with boundary conditions.

Enhanced CBO performance using adaptive region control and geometry-specific noise.

Successfully computed global minimizers for a complex constrained variational problem.

Abstract

Consensus-based optimization (CBO) is a versatile multi-particle optimization method for performing nonconvex and nonsmooth global optimizations in high dimensions. Proofs of global convergence in probability have been achieved for a broad class of objective functions in unconstrained optimizations. In this work we adapt the algorithm for solving constrained optimizations on compact and unbounded domains with boundary by leveraging emerging reflective boundary conditions. In particular, we close a relevant gap in the literature by providing a global convergence proof for the many-particle regime comprehensive of convergence rates. On the one hand, for the sake of minimizing running cost, it is desirable to keep the number of particles small. On the other hand, reducing the number of particles implies a diminished capability of exploration of the algorithm. Hence numerical heuristics are needed to ensure convergence of CBO in the few-particle regime. In this work, we also significantly improve the convergence and complexity of CBO by utilizing an adaptive region control mechanism and by choosing geometry-specific random noise. In particular, by combining a hierarchical noise structure with a multigrid finite element method, we are able to compute global minimizers for a constrained $p$-Allen-Cahn problem with obstacles, a very challenging variational problem.