Graph Neural Thompson Sampling

TL;DR

This paper introduces GNN-TS, a graph neural network-based Thompson Sampling algorithm for online decision-making on graph data, achieving state-of-the-art regret bounds and demonstrating competitive empirical performance.

Contribution

The paper proposes GNN-TS, a novel GNN-powered Thompson Sampling algorithm for graph action bandits, with theoretical regret guarantees and scalable empirical results.

Findings

Achieves sub-linear regret of order  d7((d7d T)^{1/2})

Regret bound is independent of the number of graph nodes

Empirical results show competitive performance and scalability

Abstract

We consider an online decision-making problem with a reward function defined over graph-structured data. We formally formulate the problem as an instance of graph action bandit. We then propose \texttt{GNN-TS}, a Graph Neural Network (GNN) powered Thompson Sampling (TS) algorithm which employs a GNN approximator for estimating the mean reward function and the graph neural tangent features for uncertainty estimation. We prove that, under certain boundness assumptions on the reward function, GNN-TS achieves a state-of-the-art regret bound which is (1) sub-linear of order $\tilde{\mathcal{O}}((\tilde{d} T)^{1/2})$ in the number of interaction rounds, $T$, and a notion of effective dimension $\tilde{d}$, and (2) independent of the number of graph nodes. Empirical results validate that our proposed \texttt{GNN-TS} exhibits competitive performance and scales well on graph action bandit problems.