The Sampling Complexity of Condorcet Winner Identification in Dueling Bandits

TL;DR

This paper introduces a new method for identifying the Condorcet winner in stochastic dueling bandits that leverages the full gap matrix, providing improved sample complexity guarantees and establishing fundamental lower bounds.

Contribution

It proposes a novel identification procedure exploiting the entire gap matrix and derives tight, instance-dependent sample complexity bounds with matching lower bounds for the problem.

Findings

Improved sample complexity bounds over previous methods.

First lower bounds for Condorcet winner identification in dueling bandits.

Revealed new regimes and trade-offs in sample complexity.

Abstract

We study best-arm identification in stochastic dueling bandits under the sole assumption that a Condorcet winner exists, i.e., an arm that wins each noisy pairwise comparison with probability at least $1/2$. We introduce a new identification procedure that exploits the full gap matrix $\Delta_{i,j}=q_{i,j}-\tfrac12$ (where $q_{i,j}$ is the probability that arm $i$ beats arm $j$), rather than only the gaps between the Condorcet winner and the other arms. We derive high-probability, instance-dependent sample-complexity guarantees that (up to logarithmic factors) improve the best known ones by leveraging informative comparisons beyond those involving the winner. We complement these results with new lower bounds which, to our knowledge, are the first for Condorcet-winner identification in stochastic dueling bandits. Our lower-bound analysis isolates the intrinsic cost of locating informative entries in the gap matrix and estimating them to the required confidence, establishing the optimality of our non-asymptotic bounds. Overall, our results reveal new regimes and trade-offs in the sample complexity that are not captured by asymptotic analyses based only on the expected budget.