Simple, Robust and Optimal Ranking from Pairwise Comparisons

TL;DR

This paper demonstrates that the Copeland counting algorithm is a fast, robust, and near-optimal method for ranking items based on pairwise comparisons, applicable under broad conditions and achieving theoretical limits.

Contribution

It proves the optimality and robustness of the Copeland counting algorithm for ranking from pairwise comparisons, extending guarantees to approximate recovery and arbitrary error metrics.

Findings

Copeland algorithm is computationally efficient and faster than prior methods.

It guarantees accurate top-k identification without assumptions on comparison probabilities.

The method achieves information-theoretic optimality for ranking accuracy.

Abstract

We consider data in the form of pairwise comparisons of n items, with the goal of precisely identifying the top k items for some value of k < n, or alternatively, recovering a ranking of all the items. We analyze the Copeland counting algorithm that ranks the items in order of the number of pairwise comparisons won, and show it has three attractive features: (a) its computational efficiency leads to speed-ups of several orders of magnitude in computation time as compared to prior work; (b) it is robust in that theoretical guarantees impose no conditions on the underlying matrix of pairwise-comparison probabilities, in contrast to some prior work that applies only to the BTL parametric model; and (c) it is an optimal method up to constant factors, meaning that it achieves the information-theoretic limits for recovering the top k-subset. We extend our results to obtain sharp guarantees for approximate recovery under the Hamming distortion metric, and more generally, to any arbitrary error requirement that satisfies a simple and natural monotonicity condition.