Long-term Scientific Impact Revisited

TL;DR

This paper examines the limitations of using short-term citation data to predict long-term impact of highly-cited physics papers, highlighting the unpredictability of their citation trajectories over 50 years.

Contribution

It demonstrates that current models and algorithms fail to forecast long-term citation counts from short-term data for highly-cited papers, questioning the validity of citation counts as quality proxies.

Findings

Short-term citation patterns do not distinguish highly-cited papers.

Long-term citation trajectories are unpredictable from early data.

Citation counts are not reliable indicators of long-term impact.

Abstract

Citation based measures are widely used as quantitative proxies for subjective factors such as the importance of a paper or even the worth of individual researchers. Here we analyze the citation histories of $4669$ papers published in journals of the American Physical Society between $1960$ and $1968$ and argue that state-of-the-art models of citation dynamics and algorithms for forecasting nonstationary time series are very likely to fail to predict the long-term ($50$ years after publication) citation counts of highly-cited papers using citation data collected in a short period (say, $10$ years) after publication. This is so because those papers do not exhibit distinctive short-term citation patterns, although their long-term citation patterns clearly set them apart from the other papers. We conclude that even if one accepts that citation counts are proxies for the quality of papers, they are not useful evaluative tools since the short-term counts are not informative about the long-term counts in the case of highly-cited papers.