Not dead, just resting: The practical value of per publication citation indicators

TL;DR

Citation-based indicators are limited in accurately measuring research impact, and while they can inform policy, they should complement rather than replace peer review due to inherent flaws and practical constraints.

Contribution

This paper critically examines the practical limitations of citation indicators and argues against their universal adoption over peer review in research evaluation.

Findings

Citation indicators are inferior to peer review in assessing impact.

Universal use of citation indicators is impractical due to data and cost issues.

Restructuring education for better metrics may be costly and counterproductive.

Abstract

In the final analysis citation-based indicators are inferior to effective peer review and even peer review is flawed. It is impossible to accurately measure the value or impact of scientific research and a key task of scientometricians should be to produce figures for policy makers and others that are as informative as it is practical to make them and to ensure that users are fully aware of their limitations. Although the Abramo and D'Angelo (2016) suggestions make a lot of theoretical sense and so are a goal that is worth aiming for, it is unrealistic in practice to advocate their universal use in the contexts discussed above. This is because the indicators would still have flaws in addition to the generic limitations of citation-based indicators and would still be inadequate for replacing peer review. Thus, the expense of the data gathering does not always justify the value in practice of the extra accuracy. In the longer term, the restructuring of education needed in order to get the homogeneity necessary for genuinely comparable statistics would be too expensive and probably damaging to the research mission, in addition to being out of proportion to the likely value of any citation-based indicator.