Pairwise ranking: choice of method can produce arbitrarily different rank order

TL;DR

This paper compares three pairwise ranking methods and demonstrates that the choice of method can lead to arbitrarily different rank orders, highlighting the importance of method selection in ranking tasks.

Contribution

It proves that for any two of the three methods, different rankings can be produced from the same comparison data, revealing fundamental differences in their outcomes.

Findings

Different methods can produce arbitrarily different rankings from the same data.

The choice of ranking method significantly impacts the resulting order.

The paper discusses the geometric properties of the ranking methods.

Abstract

We examine three methods for ranking by pairwise comparison: Principal Eigenvector, HodgeRank and Tropical Eigenvector. It is shown that the choice of method can produce arbitrarily different rank order.To be precise, for any two of the three methods, and for any pair of rankings of at least four items, there exists a comparison matrix for the items such that the rankings found by the two methods are the prescribed ones. We discuss the implications of this result in practice, study the geometry of the methods, and state some open problems.