Batched Online Contextual Sparse Bandits with Sequential Inclusion of Features

TL;DR

This paper introduces OBSI, a novel algorithm for batched online contextual bandits that sequentially includes features based on confidence, improving decision accuracy and fairness in sparse, real-time environments.

Contribution

The paper proposes the OBSI algorithm, which adaptively includes features in batched online bandits to enhance fairness and performance in sparse, real-time settings.

Findings

OBSI outperforms existing algorithms in regret minimization.

OBSI effectively excludes irrelevant features, improving relevance.

OBSI reduces computational costs in experiments.

Abstract

Multi-armed Bandits (MABs) are increasingly employed in online platforms and e-commerce to optimize decision making for personalized user experiences. In this work, we focus on the Contextual Bandit problem with linear rewards, under conditions of sparsity and batched data. We address the challenge of fairness by excluding irrelevant features from decision-making processes using a novel algorithm, Online Batched Sequential Inclusion (OBSI), which sequentially includes features as confidence in their impact on the reward increases. Our experiments on synthetic data show the superior performance of OBSI compared to other algorithms in terms of regret, relevance of features used, and compute.