Practical Calculation of Gittins Indices for Multi-armed Bandits

TL;DR

This paper introduces an accessible methodology and open-source tools for calculating Gittins indices in multi-armed bandit problems, making their optimal solutions more practical for common reward distributions.

Contribution

It provides a general, easy-to-implement method for computing Gittins indices, including detailed cases for Bernoulli and Gaussian rewards, reducing computational barriers.

Findings

Developed a practical calculation method for Gittins indices.

Provided open-source software for implementation.

Demonstrated applicability to Bernoulli and Gaussian reward cases.

Abstract

Gittins indices provide an optimal solution to the classical multi-armed bandit problem. An obstacle to their use has been the common perception that their computation is very difficult. This paper demonstrates an accessible general methodology for the calculating Gittins indices for the multi-armed bandit with a detailed study on the cases of Bernoulli and Gaussian rewards. With accompanying easy-to-use open source software, this work removes computation as a barrier to using Gittins indices in these commonly found settings.