lil'HDoC: An Algorithm for Good Arm Identification under Small Threshold Gap

TL;DR

This paper introduces lil'HDoC, an algorithm that improves sample efficiency for good arm identification in bandit problems with small reward-threshold gaps, outperforming existing methods in experiments.

Contribution

The paper proposes lil'HDoC, a novel algorithm that reduces sample complexity for GAI under small threshold gaps, improving upon the HDoC algorithm.

Findings

Outperforms state-of-the-art algorithms in synthetic datasets.

Reduces total sample complexity in small-gap scenarios.

Validated effectiveness on real-world datasets.

Abstract

Good arm identification (GAI) is a pure-exploration bandit problem in which a single learner outputs an arm as soon as it is identified as a good arm. A good arm is defined as an arm with an expected reward greater than or equal to a given threshold. This paper focuses on the GAI problem under a small threshold gap, which refers to the distance between the expected rewards of arms and the given threshold. We propose a new algorithm called lil'HDoC to significantly improve the total sample complexity of the HDoC algorithm. We demonstrate that the sample complexity of the first $\lambda$ output arm in lil'HDoC is bounded by the original HDoC algorithm, except for one negligible term, when the distance between the expected reward and threshold is small. Extensive experiments confirm that our algorithm outperforms the state-of-the-art algorithms in both synthetic and real-world datasets.