LAGO: A Local-Global Optimization Framework Combining Trust Region Methods and Bayesian Optimization

TL;DR

LAGO is a hybrid optimization framework that combines Bayesian Optimization with trust region methods to efficiently explore and exploit the search space, improving global exploration and local convergence for smooth functions.

Contribution

The paper introduces LAGO, a novel algorithm that integrates gradient-enhanced Bayesian Optimization with trust region local refinement using an adaptive mechanism, enhancing global and local search capabilities.

Findings

Outperforms standard local optimization algorithms in exploring the design space.

Achieves fast local convergence in promising regions.

Balances global exploration with local refinement effectively.

Abstract

We introduce LAGO, a LocAl-Global Optimization algorithm that combines gradient-enhanced Bayesian Optimization (BO) with gradient-based trust region local refinement through an adaptive competition mechanism. At each iteration, global and local optimization strategies independently propose candidate points, and the next evaluation is selected based on predicted improvement. LAGO separates global exploration from local refinement at the proposal level: the BO acquisition function is optimized outside the active trust region, while local function and gradient evaluations are incorporated into the global gradient-enhanced Gaussian process only when they satisfy a lengthscale-based minimum-distance criterion, reducing the risk of numerical instability during the local exploitation. This enables efficient local refinement when reaching promising regions, without sacrificing a global search of the design space. As a result, the method achieves an improved exploration of the full design space compared to standard non-linear local optimization algorithms for smooth functions, while maintaining fast local convergence in regions of interest.