Auditing citation polarization during the early COVID-19 pandemic

TL;DR

This study analyzes how COVID-19 research publications affected journal impact factors and citation polarization, revealing increased inequality and cautioning against overreliance on impact factor as a measure of individual paper significance.

Contribution

It provides a comprehensive quantitative analysis of COVID-19 publication citations, highlighting increased polarization and the limitations of impact factor as a quality metric.

Findings

COVID-19 research increased journal impact factor inflation.

Highly cited COVID-19 studies appeared in prestigious journals.

Impact factor reflects social attention, not individual paper quality.

Abstract

The recent pandemic stimulated scientists to publish a significant amount of research that created a surge of citations of COVID-19-related publications in a short time, leading to an abrupt inflation of the journal impact factor (IF). By auditing the complete set of COVID-19-related publications in the Web of Science, we reveal here that COVID-19-related research worsened the polarization of academic journals: the IF before the pandemic was proportional to the increment of IF, which had the effect of increasing inequality while retaining the journal rankings. We also found that the most highly cited studies related to COVID-19 were published in prestigious journals at the onset of the epidemic. Through the present quantitative investigation, our findings caution against the belief that quantitative metrics, particularly IF, can indicate the significance of individual papers. Rather, such metrics reflect the social attention given to a particular study.