Choosing to Rank

TL;DR

This paper introduces flexible choice-based models for ranking data, leveraging recent advances in discrete choice modeling to better handle complexities like multimodality and intransitivity, and demonstrates improved performance over traditional models.

Contribution

It characterizes choice representations that produce well-normalized ranking distributions and shows how to construct models with better likelihood on diverse ranking tasks.

Findings

Models outperform Plackett-Luce and Mallows in likelihood on various datasets.

Only specific variations of repeated selection admit unit normalization.

Choice-based ranking models are more flexible and accurate.

Abstract

Ranking data arises in a wide variety of application areas but remains difficult to model, learn from, and predict. Datasets often exhibit multimodality, intransitivity, or incomplete rankings---particularly when generated by humans---yet popular probabilistic models are often too rigid to capture such complexities. In this work we leverage recent progress on similar challenges in discrete choice modeling to form flexible and tractable choice-based models for ranking data. We study choice representations, maps from rankings (complete or top-$k$) to collections of choices, as a way of forming ranking models from choice models. We focus on the repeated selection (RS) choice representation, first used to form the Plackett-Luce ranking model from the conditional multinomial logit choice model. We fully characterize, for a prime number of alternatives, the choice representations that admit ranking distributions with unit normalization, a desirably property that greatly simplifies maximum likelihood estimation. We further show that only specific minor variations on repeated selection exhibit this property. Our choice-based ranking models provide higher out-of-sample likelihood when compared to Plackett-Luce and Mallows models on a broad collection of ranking tasks including food preferences, ranked-choice elections, car racing, and search engine relevance tasks.