Rank Aggregation Using Scoring Rules

TL;DR

This paper compares three methods of rank aggregation using scoring rules, analyzing their practical differences, computational complexity, and effectiveness in identifying true rankings.

Contribution

It introduces and empirically evaluates three distinct rank aggregation methods, linking them to well-known voting rules and analyzing their computational complexity.

Findings

Sequential winner selection best detects true rankings

Different methods produce significantly different rankings in practice

Complexity varies with number of voters and candidates

Abstract

To aggregate rankings into a social ranking, one can use scoring systems such as Plurality, Veto, and Borda. We distinguish three types of methods: ranking by score, ranking by repeatedly choosing a winner that we delete and rank at the top, and ranking by repeatedly choosing a loser that we delete and rank at the bottom. The latter method captures the frequently studied voting rules Single Transferable Vote (aka Instant Runoff Voting), Coombs, and Baldwin. In an experimental analysis, we show that the three types of methods produce different rankings in practice. We also provide evidence that sequentially selecting winners is most suitable to detect the "true" ranking of candidates. For different rules in our classes, we then study the (parameterized) computational complexity of deciding in which positions a given candidate can appear in the chosen ranking. As part of our analysis, we also consider the Winner Determination problem for STV, Coombs, and Baldwin and determine their complexity when there are few voters or candidates.