Linear Bandits with Partially Observable Features

TL;DR

This paper introduces a new algorithm for linear bandit problems with partially observable features, achieving sublinear regret without prior knowledge of unobserved features, and demonstrating superior performance through experiments.

Contribution

The paper proposes a novel theoretical framework and an algorithm that handles partially observable features in linear bandits with sublinear regret guarantees.

Findings

Achieves regret bound of tenilde;O(\,d + d_h)T)

Requires no prior knowledge of unobserved feature space

Outperforms non-contextual and purely observed feature-based algorithms

Abstract

We study the linear bandit problem that accounts for partially observable features. Without proper handling, unobserved features can lead to linear regret in the decision horizon $T$, as their influence on rewards is unknown. To tackle this challenge, we propose a novel theoretical framework and an algorithm with sublinear regret guarantees. The core of our algorithm consists of (i) feature augmentation, by appending basis vectors that are orthogonal to the row space of the observed features; and (ii) the introduction of a doubly robust estimator. Our approach achieves a regret bound of $\tilde{O}(\sqrt{(d + d_h)T})$, where $d$ is the dimension of the observed features and $d_h$ depends on the extent to which the unobserved feature space is contained in the observed one, thereby capturing the intrinsic difficulty of the problem. Notably, our algorithm requires no prior knowledge of the unobserved feature space, which may expand as more features become hidden. Numerical experiments confirm that our algorithm outperforms both non-contextual multi-armed bandits and linear bandit algorithms depending solely on observed features.