Turning the tables in citation analysis one more time: Principles for comparing sets of documents

TL;DR

This paper introduces new citation impact indicators based on percentile ranks instead of averages, addressing skewed citation distributions and enabling more accurate impact assessment across different document sets.

Contribution

The paper proposes a novel percentile rank-based framework for citation analysis, improving upon average-based indicators and allowing flexible comparison of document sets.

Findings

Percentile rank indicators better account for citation distribution shapes.

The proposed indicators outperform average-based metrics in skewed citation data.

Application to medical research data demonstrates their practical utility.

Abstract

We submit newly developed citation impact indicators based not on arithmetic averages of citations but on percentile ranks. Citation distributions are-as a rule-highly skewed and should not be arithmetically averaged. With percentile ranks, the citation of each paper is rated in terms of its percentile in the citation distribution. The percentile ranks approach allows for the formulation of a more abstract indicator scheme that can be used to organize and/or schematize different impact indicators according to three degrees of freedom: the selection of the reference sets, the evaluation criteria, and the choice of whether or not to define the publication sets as independent. Bibliometric data of seven principal investigators (PIs) of the Academic Medical Center of the University of Amsterdam is used as an exemplary data set. We demonstrate that the proposed indicators [R(6), R(100), R(6,k), R(100,k)] are an improvement of averages-based indicators because one can account for the shape of the distributions of citations over papers.