# Gittins' theorem under uncertainty

## TL;DR

This paper extends Gittins' theorem to uncertain environments using nonlinear expectations, demonstrating that a Gittins index remains optimal under certain independence and relaxation conditions, with insights into exploration and uncertainty aversion.

## Contribution

It introduces a framework for applying Gittins' theorem under uncertainty using nonlinear expectations and explores the impact of uncertainty on optimal decision-making.

## Key findings

- Gittins index remains optimal under strong independence and relaxed optimality conditions.
- Uncertainty influences the exploration-exploitation trade-off.
- Numerical example illustrates the effect of uncertainty aversion on decisions.

## Abstract

We study dynamic allocation problems for discrete time multi-armed bandits under uncertainty, based on the the theory of nonlinear expectations. We show that, under strong independence of the bandits and with some relaxation in the definition of optimality, a Gittins allocation index gives optimal choices. This involves studying the interaction of our uncertainty with controls which determine the filtration. We also run a simple numerical example which illustrates the interaction between the willingness to explore and uncertainty aversion of the agent when making decisions.