Stochastically Transitive Models for Pairwise Comparisons: Statistical and Computational Issues

TL;DR

This paper introduces a flexible, non-parametric model for pairwise comparison data based on stochastic transitivity, which generalizes classical models like BTL and Thurstone, and explores statistical and computational methods for estimation.

Contribution

It develops a broader class of models for pairwise comparisons, analyzes their statistical properties, and proposes computational algorithms that achieve optimal estimation rates.

Findings

Matrix of probabilities can be estimated at the same rate as in parametric models.

Singular value thresholding is consistent but not minimax optimal.

Algorithms are proposed that attain the minimax rate in certain subclasses.

Abstract

There are various parametric models for analyzing pairwise comparison data, including the Bradley-Terry-Luce (BTL) and Thurstone models, but their reliance on strong parametric assumptions is limiting. In this work, we study a flexible model for pairwise comparisons, under which the probabilities of outcomes are required only to satisfy a natural form of stochastic transitivity. This class includes parametric models including the BTL and Thurstone models as special cases, but is considerably more general. We provide various examples of models in this broader stochastically transitive class for which classical parametric models provide poor fits. Despite this greater flexibility, we show that the matrix of probabilities can be estimated at the same rate as in standard parametric models. On the other hand, unlike in the BTL and Thurstone models, computing the minimax-optimal estimator in the stochastically transitive model is non-trivial, and we explore various computationally tractable alternatives. We show that a simple singular value thresholding algorithm is statistically consistent but does not achieve the minimax rate. We then propose and study algorithms that achieve the minimax rate over interesting sub-classes of the full stochastically transitive class. We complement our theoretical results with thorough numerical simulations.