Expected values in percentile indicators

TL;DR

This paper empirically tests whether the expected proportion of highly cited papers (PP(top x%)) matches the actual observed proportions when sampling from a citation database, revealing deviations from the theoretical expectation.

Contribution

It provides an empirical evaluation of the expected values of percentile-based citation indicators and quantifies deviations through random sampling.

Findings

Deviations from expected PP(top x%) values are observed in empirical samples.

Sampling variability affects the accuracy of percentile indicators.

Results inform the reliability of using PP(top x%) in research assessment.

Abstract

PP(top x%) is the proportion of papers of a unit (e.g. an institution or a group of researchers), which belongs to the x% most frequently cited papers in the corresponding fields and publication years. It has been proposed that x% of papers can be expected which belongs to the x% most frequently cited papers. In this Letter to the Editor we will present the results of an empirical test whether we can really have this expectation and how strong the deviations from the expected values are when many random samples are drawn from the database.