Minimax Rate-Optimal Algorithms for High-Dimensional Stochastic Linear Bandits

TL;DR

This paper develops minimax rate-optimal algorithms for high-dimensional stochastic linear bandits with sparse parameters, demonstrating their superior performance over existing methods through theoretical regret bounds.

Contribution

It introduces a three-stage algorithm using thresholded Lasso for high-dimensional bandits, achieving minimax optimal regret bounds up to a logarithmic factor.

Findings

Thresholded Lasso achieves minimax rate in single-arm estimation.

The proposed bandit algorithm attains near-optimal regret bounds.

The method outperforms standard Lasso in sequential estimation.

Abstract

We study the stochastic linear bandit problem with multiple arms over $T$ rounds, where the covariate dimension $d$ may exceed $T$, but each arm-specific parameter vector is $s$-sparse. We begin by analyzing the sequential estimation problem in the single-arm setting, focusing on cumulative mean-squared error. We show that Lasso estimators are provably suboptimal in the sequential setting, exhibiting suboptimal dependence on $d$ and $T$, whereas thresholded Lasso estimators -- obtained by applying least squares to the support selected by thresholding an initial Lasso estimator -- achieve the minimax rate. Building on these insights, we consider the full linear contextual bandit problem and propose a three-stage arm selection algorithm that uses thresholded Lasso as the main estimation method. We derive an upper bound on the cumulative regret of order $s(\log s)(\log d + \log T)$, and establish a matching lower bound up to a $\log s$ factor, thereby characterizing the minimax regret rate up to a logarithmic term in $s$. Moreover, when a short initial period is excluded from the regret, the proposed algorithm achieves exact minimax optimality.