Rank Aggregation via Heterogeneous Thurstone Preference Models

TL;DR

This paper introduces the Heterogeneous Thurstone Model (HTM) for rank aggregation that accounts for varying user accuracy levels, extending existing models to heterogeneous populations and providing an efficient estimation algorithm.

Contribution

The paper presents a novel HTM that generalizes Thurstone and BTL models to heterogeneous users and proposes a convergent gradient descent algorithm for joint score and accuracy estimation.

Findings

HTM outperforms existing methods on synthetic data.

The algorithm converges linearly up to a statistical error.

HTM effectively models heterogeneous user accuracy levels.

Abstract

We propose the Heterogeneous Thurstone Model (HTM) for aggregating ranked data, which can take the accuracy levels of different users into account. By allowing different noise distributions, the proposed HTM model maintains the generality of Thurstone's original framework, and as such, also extends the Bradley-Terry-Luce (BTL) model for pairwise comparisons to heterogeneous populations of users. Under this framework, we also propose a rank aggregation algorithm based on alternating gradient descent to estimate the underlying item scores and accuracy levels of different users simultaneously from noisy pairwise comparisons. We theoretically prove that the proposed algorithm converges linearly up to a statistical error which matches that of the state-of-the-art method for the single-user BTL model. We evaluate the proposed HTM model and algorithm on both synthetic and real data, demonstrating that it outperforms existing methods.