Near Instance Optimal Model Selection for Pure Exploration Linear Bandits

TL;DR

This paper addresses the problem of model selection in pure exploration linear bandits, proposing algorithms that adapt to the complexity of the true model with near optimal guarantees in fixed confidence and fixed budget settings.

Contribution

It introduces a new optimization-based approach leveraging experimental design for near instance optimal model selection in linear bandits, including fixed budget and misspecification scenarios.

Findings

Algorithms achieve near instance optimal guarantees.

New optimization problem based on experimental design.

Effective adaptation to model misspecification.

Abstract

We introduce the model selection problem in pure exploration linear bandits, where the learner needs to adapt to the instance-dependent complexity measure of the smallest hypothesis class containing the true model. We design algorithms in both fixed confidence and fixed budget settings with near instance optimal guarantees. The core of our algorithms is a new optimization problem based on experimental design that leverages the geometry of the action set to identify a near-optimal hypothesis class. Our fixed budget algorithm is developed based on a novel selection-validation procedure, which provides a new way to study the understudied fixed budget setting (even without the added challenge of model selection). We adapt our algorithms, in both fixed confidence and fixed budget settings, to problems with model misspecification.