Regional Expected Improvement for Efficient Trust Region Selection in High-Dimensional Bayesian Optimization

TL;DR

This paper introduces a new acquisition function called regional expected improvement (REI) that enhances trust-region-based Bayesian optimization in high-dimensional problems by better identifying promising regions, leading to improved optimization performance.

Contribution

The paper proposes REI, a novel acquisition function that improves trust-region-based Bayesian optimization in high-dimensional spaces without relying on problem-specific assumptions.

Findings

REI effectively identifies regions likely to contain the global optimum.

Incorporating REI improves trust-region-based BO performance in high-dimensional problems.

REI outperforms conventional BO and other methods in real-world high-dimensional tasks.

Abstract

Real-world optimization problems often involve complex objective functions with costly evaluations. While Bayesian optimization (BO) with Gaussian processes is effective for these challenges, it suffers in high-dimensional spaces due to performance degradation from limited function evaluations. To overcome this, simplification techniques like dimensionality reduction have been employed, yet they often rely on assumptions about the problem characteristics, potentially underperforming when these assumptions do not hold. Trust-region-based methods, which avoid such assumptions, focus on local search but risk stagnation in local optima. In this study, we propose a novel acquisition function, regional expected improvement (REI), designed to enhance trust-region-based BO in medium to high-dimensional settings. REI identifies regions likely to contain the global optimum, improving performance without relying on specific problem characteristics. We provide a theoretical proof that REI effectively identifies optimal trust regions and empirically demonstrate that incorporating REI into trust-region-based BO outperforms conventional BO and other high-dimensional BO methods in medium to high-dimensional real-world problems.