Best Arm Identification in Graphical Bilinear Bandits

TL;DR

This paper introduces a new graphical bilinear bandit problem where a central entity allocates arms to graph nodes to maximize bilinear rewards, proposing a decentralized strategy with theoretical guarantees and analyzing the impact of graph structure.

Contribution

The paper formulates a novel graphical bilinear bandit problem and develops a decentralized allocation strategy with proven theoretical guarantees.

Findings

Graph structure influences convergence rate

Decentralized strategy effectively maximizes rewards

Empirical results confirm theoretical analysis

Abstract

We introduce a new graphical bilinear bandit problem where a learner (or a \emph{central entity}) allocates arms to the nodes of a graph and observes for each edge a noisy bilinear reward representing the interaction between the two end nodes. We study the best arm identification problem in which the learner wants to find the graph allocation maximizing the sum of the bilinear rewards. By efficiently exploiting the geometry of this bandit problem, we propose a \emph{decentralized} allocation strategy based on random sampling with theoretical guarantees. In particular, we characterize the influence of the graph structure (e.g. star, complete or circle) on the convergence rate and propose empirical experiments that confirm this dependency.