How are excellent (highly cited) papers defined in bibliometrics? A quantitative analysis of the literature

TL;DR

This study analyzes bibliometric literature to identify common methods for defining highly cited papers, revealing prevalent practices and thresholds used to categorize research excellence.

Contribution

It provides a comprehensive overview of the quantitative definitions and thresholds used in bibliometric studies to identify excellent papers.

Findings

Most studies use quantitative definitions for excellent papers.

Common thresholds include top 1%, 3%, and 10% citation ranks.

Average top citation thresholds are around 7.6% and 3%.

Abstract

As the subject of research excellence has received increasing attention (in science policy) over the last few decades, increasing numbers of bibliometric studies have been published dealing with excellent papers. However, many different methods have been used in these studies to identify excellent papers. The present quantitative analysis of the literature has been carried out in order to acquire an overview of these methods and an indication of an "average" or "most frequent" bibliometric practice. The search in the Web of Science yielded 321 papers dealing with "highly cited", "most cited", "top cited" and "most frequently cited". Of the 321 papers, 16 could not be used in this study. In around 80% of the papers analyzed in this study, a quantitative definition has been provided with which to identify excellent papers. With definitions which relate to an absolute number, either a certain number of top cited papers (58%) or papers with a minimum number of citations are selected (17%). Around 23% worked with percentile rank classes. Over these papers, there is an arithmetic average of the top 7.6% (arithmetic average) or of the top 3% (median). The top 1% is used most frequently in the papers, followed by the top 10%. With the thresholds presented in this study, in future, it will be possible to identify excellent papers based on an "average" or "most frequent" practice among bibliometricians.