Bandit Theory meets Compressed Sensing for high dimensional Stochastic Linear Bandit

TL;DR

This paper introduces a novel approach combining compressed sensing and bandit theory to efficiently handle high-dimensional stochastic linear bandit problems with sparse parameters, achieving regret bounds independent of the ambient dimension.

Contribution

It proposes new algorithms that leverage sparsity and compressed sensing techniques to attain regret bounds proportional to the sparsity level, improving over traditional methods in high-dimensional settings.

Findings

Achieves regret bounds of O(S√n) for sparse parameters

Demonstrates the effectiveness of combining compressed sensing with bandit algorithms

Provides theoretical guarantees for high-dimensional linear bandit problems

Abstract

We consider a linear stochastic bandit problem where the dimension $K$ of the unknown parameter $\theta$ is larger than the sampling budget $n$. In such cases, it is in general impossible to derive sub-linear regret bounds since usual linear bandit algorithms have a regret in $O(K\sqrt{n})$. In this paper we assume that $\theta$ is $S-$sparse, i.e. has at most $S-$non-zero components, and that the space of arms is the unit ball for the $||.||_2$ norm. We combine ideas from Compressed Sensing and Bandit Theory and derive algorithms with regret bounds in $O(S\sqrt{n})$.