Linear Contextual Bandits with Interference

TL;DR

This paper introduces a framework for addressing interference in linear contextual bandits, providing algorithms with theoretical guarantees and demonstrating their effectiveness through simulations.

Contribution

It develops a systematic approach to model and quantify interference effects in linear contextual bandits, bridging causal inference and online decision-making.

Findings

Algorithms achieve sublinear regret bounds.

Theoretical guarantees are established for the proposed methods.

Simulations demonstrate the effectiveness of the approach.

Abstract

Interference, a key concept in causal inference, extends the reward modeling process by accounting for the impact of one unit's actions on the rewards of others. In contextual bandit (CB) settings, where multiple units are present in the same round, potential interference can significantly affect the estimation of expected rewards for different arms, thereby influencing the decision-making process. Although some prior work has explored multi-agent and adversarial bandits in interference-aware settings, the effect of interference in CB, as well as the underlying theory, remains significantly underexplored. In this paper, we introduce a systematic framework to address interference in Linear CB (LinCB), bridging the gap between causal inference and online decision-making. We propose a series of algorithms that explicitly quantify the interference effect in the reward modeling process and provide comprehensive theoretical guarantees, including sublinear regret bounds, finite sample upper bounds, and asymptotic properties. The effectiveness of our approach is demonstrated through simulations and a synthetic data generated based on MovieLens data.