Gaussian Max-Value Entropy Search for Multi-Agent Bayesian Optimization

TL;DR

This paper introduces Gaussian Max-value Entropy Search, a multi-agent Bayesian optimization algorithm that efficiently approximates mutual information using a normal distribution, enabling scalable and effective optimization with multiple agents.

Contribution

It proposes a novel Gaussian approximation method for mutual information in multi-agent BO, reducing computational complexity and enabling scalability.

Findings

Outperforms existing multi-agent BO baselines in numerical tests.

Successfully applied to real-robot source-seeking with limited noisy data.

Achieves stable source seeking with fewer observations.

Abstract

We study the multi-agent Bayesian optimization (BO) problem, where multiple agents maximize a black-box function via iterative queries. We focus on Entropy Search (ES), a sample-efficient BO algorithm that selects queries to maximize the mutual information about the maximum of the black-box function. One of the main challenges of ES is that calculating the mutual information requires computationally-costly approximation techniques. For multi-agent BO problems, the computational cost of ES is exponential in the number of agents. To address this challenge, we propose the Gaussian Max-value Entropy Search, a multi-agent BO algorithm with favorable sample and computational efficiency. The key to our idea is to use a normal distribution to approximate the function maximum and calculate its mutual information accordingly. The resulting approximation allows queries to be cast as the solution of a closed-form optimization problem which, in turn, can be solved via a modified gradient ascent algorithm and scaled to a large number of agents. We demonstrate the effectiveness of Gaussian max-value Entropy Search through numerical experiments on standard test functions and real-robot experiments on the source-seeking problem. Results show that the proposed algorithm outperforms the multi-agent BO baselines in the numerical experiments and can stably seek the source with a limited number of noisy observations on real robots.