Dynamic Control of Explore/Exploit Trade-Off In Bayesian Optimization

TL;DR

This paper introduces a heuristic for Bayesian optimization that dynamically balances exploration and exploitation, reducing hyper-parameter tuning and enhancing optimization speed and robustness on various problems.

Contribution

It proposes a simple, effective heuristic called contextual improvement for dynamic explore/exploit control, enabling one-shot optimization without additional hyper-parameters.

Findings

Improves speed of convergence in Bayesian optimization.

Enhances robustness against initial biases and local minima.

Validated on synthetic and real-world problems.

Abstract

Bayesian optimization offers the possibility of optimizing black-box operations not accessible through traditional techniques. The success of Bayesian optimization methods such as Expected Improvement (EI) are significantly affected by the degree of trade-off between exploration and exploitation. Too much exploration can lead to inefficient optimization protocols, whilst too much exploitation leaves the protocol open to strong initial biases, and a high chance of getting stuck in a local minimum. Typically, a constant margin is used to control this trade-off, which results in yet another hyper-parameter to be optimized. We propose contextual improvement as a simple, yet effective heuristic to counter this - achieving a one-shot optimization strategy. Our proposed heuristic can be swiftly calculated and improves both the speed and robustness of discovery of optimal solutions. We demonstrate its effectiveness on both synthetic and real world problems and explore the unaccounted for uncertainty in the pre-determination of search hyperparameters controlling explore-exploit trade-off.