Rank analysis of most cited publications, a new approach for research assessments

TL;DR

This paper introduces the Rk-index, a new rank-based metric for research assessment that normalizes citation data and better captures contributions to scientific progress, especially when analyzing top-cited publications.

Contribution

The paper proposes the Rk-index, a novel, simple, and effective rank-based metric for evaluating research contributions by focusing on top-cited papers and normalizing across fields.

Findings

The Rk-index correlates with top percentile publication counts.

Rank analysis improves assessment accuracy over traditional citation metrics.

Domestic and collaborative papers should be evaluated separately.

Abstract

Citation metrics are the best tools for research assessments. However, current metrics may be misleading in research systems that pursue simultaneously different goals, such as the advance of science and incremental innovations, because their publications have different citation distributions. We estimate the contribution to the progress of knowledge by studying only a limited number of the most cited papers, which are dominated by publications pursuing this progress. To field-normalize the metrics, we substitute the number of citations by the rank position of papers from one country in the global list of papers. Using synthetic series of lognormally distributed numbers, we developed the Rk-index, which is calculated from the global ranks of the 10 highest numbers in each series, and demonstrate its equivalence to the number of papers in top percentiles, P_top0.1% and P_top0.01% . In real cases, the Rk-index is simple and easy to calculate, and evaluates the contribution to the progress of knowledge better than less stringent metrics. Although further research is needed, rank analysis of the most cited papers is a promising approach for research evaluation. It is also demonstrated that, for this purpose, domestic and collaborative papers should be studied independently.