Graph Neural Network Bandits

TL;DR

This paper introduces a novel GNN-based bandit algorithm for graph-structured reward functions, achieving scalable, permutation-invariant optimization with theoretical guarantees and competitive empirical performance.

Contribution

It establishes a new connection between permutation-invariant kernels and GNTK, enabling confidence bounds and a scalable phased-elimination algorithm for graph bandits.

Findings

Achieves sublinear regret with bounds depending on GNTK's information gain

Guarantees are independent of the number of graph nodes

Demonstrates competitive empirical performance on graph domains

Abstract

We consider the bandit optimization problem with the reward function defined over graph-structured data. This problem has important applications in molecule design and drug discovery, where the reward is naturally invariant to graph permutations. The key challenges in this setting are scaling to large domains, and to graphs with many nodes. We resolve these challenges by embedding the permutation invariance into our model. In particular, we show that graph neural networks (GNNs) can be used to estimate the reward function, assuming it resides in the Reproducing Kernel Hilbert Space of a permutation-invariant additive kernel. By establishing a novel connection between such kernels and the graph neural tangent kernel (GNTK), we introduce the first GNN confidence bound and use it to design a phased-elimination algorithm with sublinear regret. Our regret bound depends on the GNTK's maximum information gain, which we also provide a bound for. While the reward function depends on all $N$ node features, our guarantees are independent of the number of graph nodes $N$. Empirically, our approach exhibits competitive performance and scales well on graph-structured domains.