Open Problem: Model Selection for Contextual Bandits

TL;DR

This paper explores the challenge of model selection in contextual bandit algorithms, aiming to determine if adaptive guarantees similar to those in supervised learning can be achieved in this setting.

Contribution

It introduces the open problem of extending model selection guarantees to contextual bandits, highlighting a key gap in current theoretical understanding.

Findings

Identifies the gap in model selection for contextual bandits

Proposes the problem as an open question in learning theory

Highlights the need for new algorithms with adaptive guarantees

Abstract

In statistical learning, algorithms for model selection allow the learner to adapt to the complexity of the best hypothesis class in a sequence. We ask whether similar guarantees are possible for contextual bandit learning.