On the Linear Ordering Problem and the Rankability of Data

TL;DR

This paper reviews the linear ordering problem and applies it to assess data rankability, showing how optimal rankings can predict outcomes and measure diversity among rankings in sports and rankings data.

Contribution

It introduces a method to quantify data rankability using the linear ordering problem and develops a binary program to measure diversity among optimal rankings.

Findings

Optimal rankings maximize hindsight accuracy.

Degree of linearity quantifies data rankability.

Binary program measures diversity among optimal rankings.

Abstract

In 2019, Anderson et al. proposed the concept of rankability, which refers to a dataset's inherent ability to be meaningfully ranked. In this article, we give an expository review of the linear ordering problem (LOP) and then use it to analyze the rankability of data. Specifically, the degree of linearity is used to quantify what percentage of the data aligns with an optimal ranking. In a sports context, this is analogous to the number of games that a ranking can correctly predict in hindsight. In fact, under the appropriate objective function, we show that the optimal rankings computed via the LOP maximize the hindsight accuracy of a ranking. Moreover, we develop a binary program to compute the maximal Kendall tau ranking distance between two optimal rankings, which can be used to measure the diversity among optimal rankings without having to enumerate all optima. Finally, we provide several examples from the world of sports and college rankings to illustrate these concepts and demonstrate our results.