Model inference for ranking from pairwise comparisons

TL;DR

This paper introduces an efficient Bayesian algorithm to simultaneously infer unobserved object strengths and the probabilistic function linking these strengths to comparison outcomes, demonstrated through real-world case studies.

Contribution

The paper presents a novel Bayesian method for jointly estimating object strengths and the outcome probability function from pairwise comparison data.

Findings

Algorithm effectively infers strengths and probability functions.

Method is robust across different model specifications.

Case studies validate practical applicability.

Abstract

We consider the problem of ranking objects from noisy pairwise comparisons, for example, ranking tennis players from the outcomes of matches. We follow a standard approach to this problem and assume that each object has an unobserved strength and that the outcome of each comparison depends probabilistically on the strengths of the comparands. However, we do not assume to know a priori how skills affect outcomes. Instead, we present an efficient algorithm for simultaneously inferring both the unobserved strengths and the function that maps strengths to probabilities. Despite this problem being under-constrained, we present experimental evidence that the conclusions of our Bayesian approach are robust to different model specifications. We include several case studies to exemplify the method on real-world data sets.