A Case Study of the Arbitrariness of the h-Index and the Highly-Cited-Publications Indicator

TL;DR

This paper investigates how arbitrary thresholds in the h-index and highly-cited-publications indicator affect researcher rankings, demonstrating significant variability based on threshold choices through analysis of 26 physicists' citation records.

Contribution

It provides a detailed case study showing the sensitivity of bibliometric indicators to threshold parameters, highlighting their arbitrary nature and impact on rankings.

Findings

Ranking changes are large depending on threshold values.

Variability in indicators is comparable for h-index and highly-cited-publications.

Threshold arbitrariness affects evaluation consistency.

Abstract

The arbitrariness of the h-index becomes evident, when one requires q*h instead of h citations as the threshold for the definition of the index, thus changing the size of the core of the most influential publications of a dataset. I analyze the citation records of 26 physicists in order to determine how much the prefactor q influences the ranking. Likewise, the arbitrariness of the highly-cited-publications indicator is due to the threshold value, given either as an absolute number of citations or as a percentage of highly cited papers. The analysis of the 26 citation records shows that the changes in the rankings in dependence on these thresholds are rather large and comparable with the respective changes for the h-index.