Distributed Thompson sampling under constrained communication

TL;DR

This paper introduces a distributed Thompson sampling algorithm for multi-agent Bayesian optimization that accounts for communication constraints, providing theoretical regret bounds and demonstrating improved convergence with connected communication graphs.

Contribution

It develops a novel distributed Thompson sampling method with theoretical regret bounds that depend on the communication graph structure, applicable under communication constraints.

Findings

Theoretical bounds on Bayesian regret depend on the communication graph.

Connected graphs lead to faster convergence compared to disconnected graphs.

Numerical simulations confirm the importance of graph connectivity for optimization performance.

Abstract

In Bayesian optimization, a black-box function is maximized via the use of a surrogate model. We apply distributed Thompson sampling, using a Gaussian process as a surrogate model, to approach the multi-agent Bayesian optimization problem. In our distributed Thompson sampling implementation, each agent receives sampled points from neighbors, where the communication network is encoded in a graph; each agent utilizes their own Gaussian process to model the objective function. We demonstrate theoretical bounds on Bayesian average regret and Bayesian simple regret, where the bound depends on the structure of the communication graph. Unlike in batch Bayesian optimization, this bound is applicable in cases where the communication graph amongst agents is constrained. When compared to sequential single-agent Thompson sampling, our bound guarantees faster convergence with respect to time as long as the communication graph is connected. We confirm the efficacy of our algorithm with numerical simulations on traditional optimization test functions, demonstrating the significance of graph connectivity on improving regret convergence.