Geometry-Aware Approaches for Balancing Performance and Theoretical Guarantees in Linear Bandits

TL;DR

This paper introduces a geometry-aware, data-driven approach to improve linear bandit algorithms by balancing empirical performance with theoretical guarantees, achieving minimax optimal regret.

Contribution

It proposes a novel geometric, data-driven method to adaptively correct base algorithms, ensuring optimal regret bounds while preserving empirical effectiveness.

Findings

Achieves minimax optimal regret of d\u007E ilde{}(d\u007E extcolor{black}{ ext{sqrt}}(T))

Validates approach with synthetic and real data simulations

Balances empirical performance with theoretical guarantees in linear bandits

Abstract

This paper is motivated by recent research in the $d$-dimensional stochastic linear bandit literature, which has revealed an unsettling discrepancy: algorithms like Thompson sampling and Greedy demonstrate promising empirical performance, yet this contrasts with their pessimistic theoretical regret bounds. The challenge arises from the fact that while these algorithms may perform poorly in certain problem instances, they generally excel in typical instances. To address this, we propose a new data-driven technique that tracks the geometric properties of the uncertainty ellipsoid around the main problem parameter. This methodology enables us to formulate a data-driven frequentist regret bound, which incorporates the geometric information, for a broad class of base algorithms, including Greedy, OFUL, and Thompson sampling. This result allows us to identify and ``course-correct" problem instances in which the base algorithms perform poorly. The course-corrected algorithms achieve the minimax optimal regret of order $\tilde{\mathcal{O}}(d\sqrt{T})$ for a $T$-period decision-making scenario, effectively maintaining the desirable attributes of the base algorithms, including their empirical efficacy. We present simulation results to validate our findings using synthetic and real data.