Aggregating Incomplete and Noisy Rankings

TL;DR

This paper introduces a generalized model for learning true rankings from incomplete and noisy data, providing tight bounds on sample complexity and an efficient method for computing the optimal ranking.

Contribution

It proposes a selective Mallows model for incomplete rankings, derives tight bounds on sample complexity, and offers an efficient maximum likelihood estimation method.

Findings

Tight asymptotic bounds on sample complexity for ranking recovery.

Efficient algorithm for maximum likelihood ranking estimation.

Extension of Mallows model to incomplete and noisy data.

Abstract

We consider the problem of learning the true ordering of a set of alternatives from largely incomplete and noisy rankings. We introduce a natural generalization of both the classical Mallows model of ranking distributions and the extensively studied model of noisy pairwise comparisons. Our selective Mallows model outputs a noisy ranking on any given subset of alternatives, based on an underlying Mallows distribution. Assuming a sequence of subsets where each pair of alternatives appears frequently enough, we obtain strong asymptotically tight upper and lower bounds on the sample complexity of learning the underlying complete ranking and the (identities and the) ranking of the top-k alternatives from selective Mallows rankings. Moreover, building on the work of (Braverman and Mossel, 2009), we show how to efficiently compute the maximum likelihood complete ranking from selective Mallows rankings.