Thresholded Lasso Bandit

TL;DR

This paper introduces the Thresholded Lasso bandit algorithm for sparse linear bandit problems, achieving improved regret bounds and outperforming existing methods without prior knowledge of sparsity.

Contribution

The paper presents a novel sparse bandit algorithm that estimates the reward vector and its support using thresholded Lasso, with non-asymptotic regret bounds and no need for prior sparsity knowledge.

Findings

Regret bounds scale as O(log d + sqrt(T)) and O(log d + log T) under different conditions.

Algorithm outperforms existing methods in numerical experiments.

Does not require prior knowledge of sparsity index s_0.

Abstract

In this paper, we revisit the regret minimization problem in sparse stochastic contextual linear bandits, where feature vectors may be of large dimension $d$, but where the reward function depends on a few, say $s_0\ll d$, of these features only. We present Thresholded Lasso bandit, an algorithm that (i) estimates the vector defining the reward function as well as its sparse support, i.e., significant feature elements, using the Lasso framework with thresholding, and (ii) selects an arm greedily according to this estimate projected on its support. The algorithm does not require prior knowledge of the sparsity index $s_0$ and can be parameter-free under some symmetric assumptions. For this simple algorithm, we establish non-asymptotic regret upper bounds scaling as $\mathcal{O}( \log d + \sqrt{T} )$ in general, and as $\mathcal{O}( \log d + \log T)$ under the so-called margin condition (a probabilistic condition on the separation of the arm rewards). The regret of previous algorithms scales as $\mathcal{O}( \log d + \sqrt{T \log (d T)})$ and $\mathcal{O}( \log T \log d)$ in the two settings, respectively. Through numerical experiments, we confirm that our algorithm outperforms existing methods.