Laplacian Kernelized Bandit

TL;DR

This paper introduces a novel multi-user kernel for bandit problems that combines graph Laplacian and base kernels, enabling effective exploration in non-linear, graph-structured reward settings with theoretical guarantees.

Contribution

It develops a unified multi-user RKHS kernel integrating graph Laplacian and base kernels, and designs algorithms with regret bounds that depend on an effective dimension, not user count.

Findings

Outperforms linear and non-graph baselines in non-linear settings.

Remains competitive when rewards are linear.

Provides a scalable regret bound based on effective dimension.

Abstract

We study multi-user contextual bandits where users are related by a graph and their reward functions exhibit both non-linear behavior and graph homophily. We introduce a principled joint penalty for the collection of user reward functions $\{f_u\}$, combining a graph smoothness term based on RKHS distances with an individual roughness penalty. Our central contribution is proving that this penalty is equivalent to the squared norm within a single, unified \emph{multi-user RKHS}. We explicitly derive its reproducing kernel, which elegantly fuses the graph Laplacian with the base arm kernel. This unification allows us to reframe the problem as learning a single ''lifted'' function, enabling the design of principled algorithms, \texttt{LK-GP-UCB} and \texttt{LK-GP-TS}, that leverage Gaussian Process posteriors over this new kernel for exploration. We provide high-probability regret bounds that scale with an \emph{effective dimension} of the multi-user kernel, replacing dependencies on user count or ambient dimension. Empirically, our methods outperform strong linear and non-graph-aware baselines in non-linear settings and remain competitive even when the true rewards are linear. Our work delivers a unified, theoretically grounded, and practical framework that bridges Laplacian regularization with kernelized bandits for structured exploration.