Synopsis: Sequential Decision Problems with Weak Feedback

TL;DR

This thesis explores sequential decision problems with limited or weak feedback, proposing optimal algorithms for various setups and validating their effectiveness on synthetic and real data.

Contribution

It introduces new problem setups like Censored Semi Bandits and develops provably optimal algorithms tailored for weak feedback scenarios.

Findings

Algorithms achieve optimal performance in weak feedback settings.

Empirical validation shows effectiveness on synthetic and real datasets.

Addresses diverse applications like healthcare and resource allocation.

Abstract

This thesis considers sequential decision problems, where the loss/reward incurred by selecting an action may not be inferred from observed feedback. A major part of this thesis focuses on the unsupervised sequential selection problem, where one can not infer the loss incurred for selecting an action from observed feedback. We also introduce a new setup named Censored Semi Bandits, where the loss incurred for selecting an action can be observed under certain conditions. Finally, we study the channel selection problem in the communication networks, where the reward for an action is only observed when no other player selects that action to play in the round. These problems find applications in many fields like healthcare, crowd-sourcing, security, adaptive resource allocation, among many others. This thesis aims to address the above-described sequential decision problems by exploiting specific structures these problems exhibit. We develop provably optimal algorithms for each of these setups with weak feedback and validate their empirical performance on different problem instances derived from synthetic and real datasets.