CAGES: Cost-Aware Gradient Entropy Search for Efficient Local Multi-Fidelity Bayesian Optimization

TL;DR

CAGES introduces a cost-aware, multi-fidelity local Bayesian optimization method that efficiently learns gradients in high-dimensional spaces, outperforming existing approaches on synthetic and RL benchmarks.

Contribution

It proposes a novel, flexible multi-fidelity local BO algorithm with an information-theoretic acquisition function for gradient learning, independent of source relationships.

Findings

CAGES outperforms state-of-the-art methods on synthetic benchmarks.

CAGES achieves significant improvements in RL policy search tasks.

The method efficiently balances evaluation cost and information gain.

Abstract

Bayesian optimization (BO) is a popular approach for optimizing expensive-to-evaluate black-box objective functions. An important challenge in BO is its application to high-dimensional search spaces due in large part to the curse of dimensionality. One way to overcome this challenge is to focus on local BO methods that aim to efficiently learn gradients, which have shown strong empirical performance on high-dimensional problems including policy search in reinforcement learning (RL). Current local BO methods assume access to only a single high-fidelity information source whereas, in many problems, one has access to multiple cheaper approximations of the objective. We propose a novel algorithm, Cost-Aware Gradient Entropy Search (CAGES), for local BO of multi-fidelity black-box functions. CAGES makes no assumption about the relationship between different information sources, making it more flexible than other multi-fidelity methods. It also employs a new information-theoretic acquisition function, which enables systematic identification of samples that maximize the information gain about the unknown gradient per evaluation cost. We demonstrate CAGES can achieve significant performance improvements compared to other state-of-the-art methods on synthetic and benchmark RL problems.