MaxGap Bandit: Adaptive Algorithms for Approximate Ranking

TL;DR

This paper introduces the MaxGap-bandit problem, focusing on adaptively identifying the largest gap between adjacent means in distributions, with algorithms that are proven minimax optimal and significantly reduce sampling requirements.

Contribution

It formulates the MaxGap-bandit problem, proposes optimal algorithms, and demonstrates substantial sample efficiency improvements over non-adaptive methods.

Findings

UCB-style algorithms are 6-8x more sample-efficient than non-adaptive methods.

The problem captures natural tasks like approximate ranking and noisy sorting.

Algorithms are proven to be minimax optimal.

Abstract

This paper studies the problem of adaptively sampling from K distributions (arms) in order to identify the largest gap between any two adjacent means. We call this the MaxGap-bandit problem. This problem arises naturally in approximate ranking, noisy sorting, outlier detection, and top-arm identification in bandits. The key novelty of the MaxGap-bandit problem is that it aims to adaptively determine the natural partitioning of the distributions into a subset with larger means and a subset with smaller means, where the split is determined by the largest gap rather than a pre-specified rank or threshold. Estimating an arm's gap requires sampling its neighboring arms in addition to itself, and this dependence results in a novel hardness parameter that characterizes the sample complexity of the problem. We propose elimination and UCB-style algorithms and show that they are minimax optimal. Our experiments show that the UCB-style algorithms require 6-8x fewer samples than non-adaptive sampling to achieve the same error.