Journal ranking should depend on the level of aggregation

TL;DR

This paper proves an impossibility theorem showing that any consistent journal ranking method must depend on the level of aggregation, highlighting a fundamental trade-off in ranking axioms.

Contribution

It introduces an axiomatic framework for journal ranking and demonstrates a fundamental trade-off, revealing the necessity of aggregation dependence in ranking methods.

Findings

Any self-consistent ranking method must depend on aggregation level.

There is a trade-off between axiomatic properties in journal ranking.

The result highlights a dilemma in designing ranking systems.

Abstract

Journal ranking is becoming more important in assessing the quality of academic research. Several indices have been suggested for this purpose, typically on the basis of a citation graph between the journals. We follow an axiomatic approach and find an impossibility theorem: any self-consistent ranking method, which satisfies a natural monotonicity property, should depend on the level of aggregation. Our result presents a trade-off between two axiomatic properties and reveals a dilemma of aggregation.