Double-Linear Thompson Sampling for Context-Attentive Bandits

TL;DR

This paper introduces CATS, a novel algorithm for context-attentive bandits that selectively observes context variables, with theoretical regret bounds and empirical validation showing its effectiveness over baselines.

Contribution

It extends Linear Thompson Sampling to the context-attentive setting, enabling selective observation of context variables with proven regret bounds.

Findings

CATS outperforms baseline methods on real-world datasets.

Theoretical regret bounds are established for the proposed algorithm.

Empirical results demonstrate the efficiency of context variable selection.

Abstract

In this paper, we analyze and extend an online learning framework known as Context-Attentive Bandit, motivated by various practical applications, from medical diagnosis to dialog systems, where due to observation costs only a small subset of a potentially large number of context variables can be observed at each iteration;however, the agent has a freedom to choose which variables to observe. We derive a novel algorithm, called Context-Attentive Thompson Sampling (CATS), which builds upon the Linear Thompson Sampling approach, adapting it to Context-Attentive Bandit setting. We provide a theoretical regret analysis and an extensive empirical evaluation demonstrating advantages of the proposed approach over several baseline methods on a variety of real-life datasets