Nearly Minimax Algorithms for Linear Bandits with Shared Representation

TL;DR

This paper introduces nearly minimax algorithms for multi-task and lifelong linear bandits with shared low-dimensional representations, achieving optimal regret bounds that match theoretical lower bounds up to logarithmic factors.

Contribution

The paper presents novel algorithms with improved regret bounds for shared representation linear bandits, closing the gap with minimax lower bounds and introducing a more efficient estimator.

Findings

Achieves regret bounds of (d ext{ }kMT + kM ext{ }  ext{ }T)

Matches the minimax regret lower bounds up to logarithmic factors

Introduces a more efficient estimator for low-rank linear feature extraction.

Abstract

We give novel algorithms for multi-task and lifelong linear bandits with shared representation. Specifically, we consider the setting where we play $M$ linear bandits with dimension $d$, each for $T$ rounds, and these $M$ bandit tasks share a common $k(\ll d)$ dimensional linear representation. For both the multi-task setting where we play the tasks concurrently, and the lifelong setting where we play tasks sequentially, we come up with novel algorithms that achieve $\widetilde{O}\left(d\sqrt{kMT} + kM\sqrt{T}\right)$ regret bounds, which matches the known minimax regret lower bound up to logarithmic factors and closes the gap in existing results [Yang et al., 2021]. Our main technique include a more efficient estimator for the low-rank linear feature extractor and an accompanied novel analysis for this estimator.