Bandits with Partially Observable Confounded Data

TL;DR

This paper introduces a linear contextual bandit algorithm that leverages confounded offline data to improve regret bounds, demonstrating that offline data can significantly enhance online learning performance.

Contribution

It presents a novel approach to utilize confounded offline data in linear bandits, with theoretical regret bounds and empirical validation showing improved performance.

Findings

Regret bounds improved by a factor related to visible context dimensionality

Confounded offline data can be effectively exploited for online learning

Synthetic simulations validate the approach's effectiveness

Abstract

We study linear contextual bandits with access to a large, confounded, offline dataset that was sampled from some fixed policy. We show that this problem is closely related to a variant of the bandit problem with side information. We construct a linear bandit algorithm that takes advantage of the projected information, and prove regret bounds. Our results demonstrate the ability to take advantage of confounded offline data. Particularly, we prove regret bounds that improve current bounds by a factor related to the visible dimensionality of the contexts in the data. Our results indicate that confounded offline data can significantly improve online learning algorithms. Finally, we demonstrate various characteristics of our approach through synthetic simulations.