Validated research assessment based on highly cited researchers

TL;DR

This paper validates the use of highly cited researchers as a simple, effective indicator for assessing national research success, highlighting its advantages and limitations compared to other bibliometric measures.

Contribution

It provides empirical validation for using highly cited researchers as a research assessment tool at the country level and compares different sources of such data.

Findings

Highly cited researchers correlate with research success.

Clarivate Analytics indicator has flaws in some countries.

Evaluation based on highly cited researchers is approximately equivalent to top 5% or 0.05% papers.

Abstract

Bibliometrics provides accurate, cheap and simple descriptions of research systems and should lay the foundations for research policy. However, disconnections between bibliometric knowledge and research policy frequently misguide the research policy in many countries. A way of correcting these disconnections might come from the use of simple indicators of research performance. One such simple indicator is the number of highly cited researchers, which can be used under the assumption that a research system that produces and employs many highly cited researchers will be more successful than others with fewer of them. Here, we validate the use of the number of highly cited researchers (Ioannidis et al. 2020; PLoS Biol 18(10): e3000918) for research assessment at the country level and determine a country ranking of research success. We also demonstrate that the number of highly cited researchers reported by Clarivate Analytics is also an indicator of the research success of countries. The formal difference between the numbers of highly cited researchers according to Ionannidis et al. and Clarivate Analytics is that evaluations based on these two lists of highly cited researchers are approximately equivalent to evaluations based on the top 5% and 0.05% of highly cited papers, respectively. Moreover, the Clarivate Analytics indicator is flawed in some countries.