Regret Analysis of the Finite-Horizon Gittins Index Strategy for Multi-Armed Bandits

TL;DR

This paper provides a finite-time regret analysis of the Gittins index strategy for Gaussian multi-armed bandits, showing it has regret guarantees comparable to UCB and Thompson sampling, with practical improvements.

Contribution

It offers the first finite-time regret bounds for the Gittins index strategy in Gaussian bandits and compares its performance to existing algorithms.

Findings

Gittins index strategy achieves finite-time regret bounds similar to UCB.

Finite-time bounds on the Gittins index are asymptotically exact.

Experimental results show a version of Gittins index outperforms UCB and Thompson sampling.

Abstract

I analyse the frequentist regret of the famous Gittins index strategy for multi-armed bandits with Gaussian noise and a finite horizon. Remarkably it turns out that this approach leads to finite-time regret guarantees comparable to those available for the popular UCB algorithm. Along the way I derive finite-time bounds on the Gittins index that are asymptotically exact and may be of independent interest. I also discuss some computational issues and present experimental results suggesting that a particular version of the Gittins index strategy is a modest improvement on existing algorithms with finite-time regret guarantees such as UCB and Thompson sampling.