Thresholding Bandit Problem with Both Duels and Pulls

TL;DR

This paper introduces a novel Thresholding Bandit Problem variant that incorporates both arm pulls and duels, providing an efficient algorithm with proven optimality and superior experimental performance.

Contribution

It proposes the TBP-DC setting, develops the Rank-Search algorithm, and establishes its theoretical guarantees and optimality compared to existing methods.

Findings

Rank-Search outperforms baseline algorithms in experiments.

Theoretical guarantees confirm the optimality of Rank-Search.

Dueling choices can be more cost-effective than pulls in crowdsourcing applications.

Abstract

The Thresholding Bandit Problem (TBP) aims to find the set of arms with mean rewards greater than a given threshold. We consider a new setting of TBP, where in addition to pulling arms, one can also \emph{duel} two arms and get the arm with a greater mean. In our motivating application from crowdsourcing, dueling two arms can be more cost-effective and time-efficient than direct pulls. We refer to this problem as TBP with Dueling Choices (TBP-DC). This paper provides an algorithm called Rank-Search (RS) for solving TBP-DC by alternating between ranking and binary search. We prove theoretical guarantees for RS, and also give lower bounds to show the optimality of it. Experiments show that RS outperforms previous baseline algorithms that only use pulls or duels.