Differentially Private Kernelized Contextual Bandits

TL;DR

This paper introduces a differentially private algorithm for kernelized contextual bandits that achieves improved error bounds by leveraging a novel reward estimator with low sensitivity, balancing privacy and learning accuracy.

Contribution

It proposes a new differentially private algorithm for kernelized contextual bandits with an innovative reward estimator, improving error bounds and privacy-utility trade-offs.

Findings

Achieves an error rate of O(√(γ_T/T) + γ_T/(Tε)) for large kernel classes.

Introduces a reward estimator with high utility and low sensitivity.

Provides theoretical guarantees under joint differential privacy constraints.

Abstract

We consider the problem of contextual kernel bandits with stochastic contexts, where the underlying reward function belongs to a known Reproducing Kernel Hilbert Space (RKHS). We study this problem under the additional constraint of joint differential privacy, where the agents needs to ensure that the sequence of query points is differentially private with respect to both the sequence of contexts and rewards. We propose a novel algorithm that improves upon the state of the art and achieves an error rate of $\mathcal{O}\left(\sqrt{\frac{\gamma_T}{T}} + \frac{\gamma_T}{T \varepsilon}\right)$ after $T$ queries for a large class of kernel families, where $\gamma_T$ represents the effective dimensionality of the kernel and $\varepsilon > 0$ is the privacy parameter. Our results are based on a novel estimator for the reward function that simultaneously enjoys high utility along with a low-sensitivity to observed rewards and contexts, which is crucial to obtain an order optimal learning performance with improved dependence on the privacy parameter.