Ranking Inferences Based on the Top Choice of Multiway Comparisons

TL;DR

This paper develops a statistical framework for ranking items based on top-choice data from multiway comparisons, extending existing models, establishing convergence rates, and providing confidence intervals for scores and ranks.

Contribution

It introduces a novel inference method for multiway ranking data, including asymptotic normality results and a bootstrap-based confidence interval construction.

Findings

Established convergence rates for preference scores.

Proposed a bootstrap method for confidence intervals.

Validated methods through simulations and real data.

Abstract

This paper considers ranking inference of $n$ items based on the observed data on the top choice among $M$ randomly selected items at each trial. This is a useful modification of the Plackett-Luce model for $M$-way ranking with only the top choice observed and is an extension of the celebrated Bradley-Terry-Luce model that corresponds to $M=2$. Under a uniform sampling scheme in which any $M$ distinguished items are selected for comparisons with probability $p$ and the selected $M$ items are compared $L$ times with multinomial outcomes, we establish the statistical rates of convergence for underlying $n$ preference scores using both $\ell_2$-norm and $\ell_\infty$-norm, with the minimum sampling complexity. In addition, we establish the asymptotic normality of the maximum likelihood estimator that allows us to construct confidence intervals for the underlying scores. Furthermore, we propose a novel inference framework for ranking items through a sophisticated maximum pairwise difference statistic whose distribution is estimated via a valid Gaussian multiplier bootstrap. The estimated distribution is then used to construct simultaneous confidence intervals for the differences in the preference scores and the ranks of individual items. They also enable us to address various inference questions on the ranks of these items. Extensive simulation studies lend further support to our theoretical results. A real data application illustrates the usefulness of the proposed methods convincingly.