Graph Resistance and Learning from Pairwise Comparisons

TL;DR

This paper analyzes how the structure of a comparison graph influences the accuracy of estimating item qualities from noisy pairwise comparisons, introducing a minimax optimal algorithm with error bounds tied to graph resistance.

Contribution

It establishes a new connection between graph resistance and estimation error, providing an optimal algorithm with improved guarantees over previous methods.

Findings

Error scales with the square root of graph resistance

Algorithm is minimax optimal up to log factors

Performance surpasses earlier results in accuracy and robustness

Abstract

We consider the problem of learning the qualities of a collection of items by performing noisy comparisons among them. Following the standard paradigm, we assume there is a fixed "comparison graph" and every neighboring pair of items in this graph is compared $k$ times according to the Bradley-Terry-Luce model (where the probability than an item wins a comparison is proportional the item quality). We are interested in how the relative error in quality estimation scales with the comparison graph in the regime where $k$ is large. We prove that, after a known transition period, the relevant graph-theoretic quantity is the square root of the resistance of the comparison graph. Specifically, we provide an algorithm that is minimax optimal. The algorithm has a relative error decay that scales with the square root of the graph resistance, and provide a matching lower bound (up to log factors). The performance guarantee of our algorithm, both in terms of the graph and the skewness of the item quality distribution, outperforms earlier results.