Bandit Problems with Side Observations

TL;DR

This paper extends the two-armed bandit problem by incorporating side observations, analyzing how additional information influences decision-making, and developing optimal strategies with theoretical bounds.

Contribution

It introduces a framework for bandit problems with side observations, deriving lower bounds and constructing asymptotically optimal adaptive algorithms.

Findings

Lower bounds on inferior sampling time with side information

Optimal adaptive algorithms matching these bounds

Quantification of the benefit of side observations

Abstract

An extension of the traditional two-armed bandit problem is considered, in which the decision maker has access to some side information before deciding which arm to pull. At each time t, before making a selection, the decision maker is able to observe a random variable X_t that provides some information on the rewards to be obtained. The focus is on finding uniformly good rules (that minimize the growth rate of the inferior sampling time) and on quantifying how much the additional information helps. Various settings are considered and for each setting, lower bounds on the achievable inferior sampling time are developed and asymptotically optimal adaptive schemes achieving these lower bounds are constructed.