Distributed Linear Bandits under Communication Constraints

TL;DR

This paper studies distributed linear bandit learning under communication limits, establishing fundamental bounds and proposing algorithms that optimize the trade-off between regret minimization and communication cost, including for sparse cases.

Contribution

It provides the first information-theoretic lower bounds on communication for sublinear regret and develops algorithms that achieve these bounds, improving distributed bandit efficiency.

Findings

Established lower bounds on communication for sublinear regret

Designed algorithms matching the optimal regret-communication trade-off

Extended results to sparse linear bandits with improved trade-offs

Abstract

We consider distributed linear bandits where $M$ agents learn collaboratively to minimize the overall cumulative regret incurred by all agents. Information exchange is facilitated by a central server, and both the uplink and downlink communications are carried over channels with fixed capacity, which limits the amount of information that can be transmitted in each use of the channels. We investigate the regret-communication trade-off by (i) establishing information-theoretic lower bounds on the required communications (in terms of bits) for achieving a sublinear regret order; (ii) developing an efficient algorithm that achieves the minimum sublinear regret order offered by centralized learning using the minimum order of communications dictated by the information-theoretic lower bounds. For sparse linear bandits, we show a variant of the proposed algorithm offers better regret-communication trade-off by leveraging the sparsity of the problem.