Single Index Bandits: Generalized Linear Contextual Bandits with Unknown Reward Functions

TL;DR

This paper introduces new algorithms for generalized linear bandits with unknown reward functions, addressing the practical challenge of link function misspecification and achieving near-optimal regret bounds.

Contribution

It proposes the STOR, ESTOR, and GSTOR algorithms for unknown and general reward functions, extending bandit theory to more realistic scenarios.

Findings

ESTOR achieves nearly optimal $ ilde{O}_T( oot{T})$ regret.

Algorithms perform well on synthetic and real datasets.

Extensions to high-dimensional sparse settings are effective.

Abstract

Generalized linear bandits have been extensively studied due to their broad applicability in real-world online decision-making problems. However, these methods typically assume that the expected reward function is known to the users, an assumption that is often unrealistic in practice. Misspecification of this link function can lead to the failure of all existing algorithms. In this work, we address this critical limitation by introducing a new problem of generalized linear bandits with unknown reward functions, also known as single index bandits. We first consider the case where the unknown reward function is monotonically increasing, and propose two novel and efficient algorithms, STOR and ESTOR, that achieve decent regrets under standard assumptions. Notably, our ESTOR can obtain the nearly optimal regret bound $\tilde{O}_T(\sqrt{T})$ in terms of the time horizon $T$. We then extend our methods to the high-dimensional sparse setting and show that the same regret rate can be attained with the sparsity index. Next, we introduce GSTOR, an algorithm that is agnostic to general reward functions, and establish regret bounds under a Gaussian design assumption. Finally, we validate the efficiency and effectiveness of our algorithms through experiments on both synthetic and real-world datasets.