Sequential Decision Problems with Weak Feedback

TL;DR

This thesis explores sequential decision problems with limited feedback, introducing new models and algorithms for unsupervised and censored feedback scenarios, with applications across various fields.

Contribution

It introduces a new setup called Censored Semi Bandits and develops optimal algorithms for weak feedback problems, validated on synthetic and real data.

Findings

Developed provably optimal algorithms for weak feedback scenarios.

Validated algorithms' empirical performance on diverse datasets.

Extended understanding of decision-making with limited feedback.

Abstract

This thesis considers sequential decision problems, where the loss/reward incurred by selecting an action may not be inferred from observed feedback. A major part of this thesis focuses on the unsupervised sequential selection problem, where one can not infer the loss incurred for selecting an action from observed feedback. We also introduce a new setup named Censored Semi Bandits, where the loss incurred for selecting an action can be observed under certain conditions. Finally, we study the channel selection problem in the communication networks, where the reward for an action is only observed when no other player selects that action to play in the round. These problems find applications in many fields like healthcare, crowd-sourcing, security, adaptive resource allocation, among many others. This thesis aims to address the above-described sequential decision problems by exploiting specific structures these problems exhibit. We develop provably optimal algorithms for each of these setups with weak feedback and validate their empirical performance on different problem instances derived from synthetic and real datasets.