Multi-armed Bandit Problem with Known Trend

TL;DR

This paper introduces a variant of the multi-armed bandit problem where the reward trend is known, proposing an adapted algorithm that leverages this information to improve decision-making in online applications.

Contribution

It presents the A-UCB algorithm tailored for known reward trends, with theoretical regret bounds and experimental validation demonstrating its effectiveness.

Findings

A-UCB outperforms standard UCB1 in known trend settings.

Theoretical regret bounds are improved over traditional methods.

Experimental results confirm the advantages of the proposed approach.

Abstract

We consider a variant of the multi-armed bandit model, which we call multi-armed bandit problem with known trend, where the gambler knows the shape of the reward function of each arm but not its distribution. This new problem is motivated by different online problems like active learning, music and interface recommendation applications, where when an arm is sampled by the model the received reward change according to a known trend. By adapting the standard multi-armed bandit algorithm UCB1 to take advantage of this setting, we propose the new algorithm named A-UCB that assumes a stochastic model. We provide upper bounds of the regret which compare favourably with the ones of UCB1. We also confirm that experimentally with different simulations