The Gittins Index: A Design Principle for Decision-Making Under Uncertainty

TL;DR

This paper demonstrates that the Gittins index, traditionally viewed as a theoretical concept, can be effectively applied to practical decision-making problems like Bayesian optimization and queue latency reduction, showing promising results.

Contribution

It provides an example-driven tutorial showing practical applications of the Gittins index beyond its theoretical origins, including case studies in Bayesian optimization and queue management.

Findings

Gittins index can be applied to practical problems with excellent performance.

It achieves near-optimal results in several decision-making scenarios.

The tutorial bridges theory and practice for the Gittins index.

Abstract

The Gittins index is a tool that optimally solves a variety of decision-making problems involving uncertainty, including multi-armed bandit problems, minimizing mean latency in queues, and search problems like the Pandora's box model. However, despite the above examples and later extensions thereof, the space of problems that the Gittins index can solve perfectly optimally is limited, and its definition is rather subtle compared to those of other multi-armed bandit algorithms. As a result, the Gittins index is often regarded as being primarily a concept of theoretical importance, rather than a practical tool for solving decision-making problems. The aim of this tutorial is to demonstrate that the Gittins index can be fruitfully applied to practical problems. We start by giving an example-driven introduction to the Gittins index, then walk through several examples of problems it solves - some optimally, some suboptimally but still with excellent performance. Two practical highlights in the latter category are applying the Gittins index to Bayesian optimization, and applying the Gittins index to minimizing tail latency in queues.