Sum of rank ratios: an alternative to percentiles for research assessment, from groundbreaking to mainstream research

TL;DR

This paper introduces the Rn index, a rank-based metric for assessing groundbreaking research, which outperforms traditional citation-based indicators and is easy to compute.

Contribution

The study proposes the Rn index, a novel, simple, and effective method for evaluating high-impact research, especially for infrequent, boundary-pushing outputs.

Findings

Rn index outperforms top percentile citation indicators.

It effectively evaluates countries with less frequent groundbreaking research.

The Rn index is easy to calculate and robust to citation ties.

Abstract

Assessing research that pushes the boundaries of knowledge is challenging because such work is extremely infrequent, accounting for only about 0.01 per cent of all research outputs. Consequently, knowledge about how to evaluate this type of research is far more limited than the well established methods used to assess more common research outcomes. This study addresses this gap by using a rank based approach in which each paper is assigned a unique value equal to the ratio between its local and global ranks. The cumulative value of these ratios, starting from the most cited paper, provides the evaluative basis, and the Rn index described here, using 10 rank ratios, appears to be the best option. Although research assessment based on global ranks was originally developed to evaluate the largest contributors to groundbreaking knowledge, namely, the USA and China, which account for most of the most cited papers, the Rn index has broader applications. This study demonstrates that it is also a better option than the number of top 10 per cent or top 1per cent highly cited papers, which are the most common indicators used to evaluate countries that seldom or never produce cutting-edge research that pushes the boundaries of knowledge. In all cases, the Rn index reflects the highest quality science produced by each country. Furthermore, the Rn index can be easily calculated without specialized training in bibliometrics and is insignificantly affected by ties in citation counts.