Informed peer review for publication assessments: Are improved impact measures worth the hassle?

TL;DR

This paper evaluates whether combining early citations with journal impact factors improves research performance assessments of Italian scientists, revealing significant field-specific shifts and suggesting benefits of the combined approach especially with many uncited researchers.

Contribution

It introduces a method that combines early citations and impact factors to assess research impact, analyzing its effects on rankings across multiple scientific fields.

Findings

Strong correlation between citation-based and combined impact scores.

Significant ranking shifts observed in Economics, Mathematics, and Computer Science.

Combined indicator is more advisable when many researchers are uncited or citation windows are short.

Abstract

In this work we ask whether and to what extent applying a predictor of publications' impact better than early citations, has an effect on the assessment of research performance of individual scientists. Specifically, we measure the total impact of Italian professors in the sciences and economics in a period of time, valuing their publications first by early citations and then by a weighted combination of early citations and impact factor of the hosting journal. As expected, scores and ranks by the two indicators show a very strong correlation, but there occur also significant shifts in many fields, mainly in Economics and statistics, and Mathematics and computer science. The higher the share of uncited professors in a field and the shorter the citation time window, the more recommendable the recourse to the above combination.