Beyond Objective-Based Improvement: Stationarity-Aware Expected Improvement for Bayesian Optimization

TL;DR

This paper introduces EI-GN, a new Bayesian Optimization acquisition function that incorporates stationarity to improve sampling efficiency, demonstrating better performance on benchmarks and control policy learning.

Contribution

It proposes EI-GN, a novel acquisition function that combines objective improvement with stationarity considerations, extending traditional Expected Improvement methods.

Findings

EI-GN outperforms baseline methods on standard BO benchmarks.

EI-GN effectively guides sampling toward high-performing, stationary regions.

The method is applicable to control policy learning.

Abstract

Bayesian Optimization (BO) is a principled framework for optimizing expensive black-box functions, with Expected Improvement (EI) among its most widely used acquisition functions. Despite its empirical success, EI is agnostic to first-order optimality conditions, relying solely on objective-value improvement. As a result, it can exhibit vanishing acquisition signals where the improvement criterion is uninformative, limiting its effectiveness in guiding search. We propose Expected Improvement via Gradient Norms (EI-GN), a novel acquisition function that extends the improvement principle to incorporate first-order stationarity, promoting sampling in regions that are both high-performing and close to stationary points. We derive a tractable closed-form expression for EI-GN and show that it remains consistent with the improvement-based acquisition framework. By embedding progress toward stationarity into the acquisition criterion, EI-GN provides a richer and more informative notion of improvement. Empirical results on standard BO benchmarks demonstrate consistent gains over baseline methods, and we further illustrate its applicability to control policy learning.