How much do different ways of calculating percentiles influence the derived performance indicators? - A case study

TL;DR

This study examines how different methods of calculating percentiles affect bibliometric performance indicators, highlighting potential biases caused by handling tied citation counts at percentile thresholds.

Contribution

It provides a case study demonstrating the impact of various percentile calculation methods on bibliometric evaluations, emphasizing the importance of consistent tie-breaking procedures.

Findings

Different percentile calculation methods lead to significant deviations in performance indicators.

Treating tied papers at thresholds differently can introduce strong biases.

The case study highlights the need for standardized approaches in bibliometric assessments.

Abstract

Bibliometric indicators can be determined by comparing specific citation records with the percentiles of a reference set. However, there exists an ambiguity in the computation of percentiles because usually a significant number of papers with the same citation count are found at the border between percentile rank classes. The present case study of the citations to the journal Europhysics Letters (EPL) in comparison with all physics papers from the Web of Science shows the deviations which occur due to the different ways of treating the tied papers in the evaluation of the percentage of highly cited publications. A strong bias can occur, if the papers tied at the threshold number of citations are all considered as highly cited or all considered as not highly cited.