Peer review vs bibliometrics: which method better predicts the scholarly impact of publications?

TL;DR

This study compares peer review and bibliometric indicators in predicting the long-term scholarly impact of publications, finding that early citation metrics outperform peer review evaluations across disciplines.

Contribution

It provides empirical evidence that early citation-based indicators are more effective than peer review in forecasting future citations of scientific publications.

Findings

Early citation indicators better predict long-term impact.

Bibliometric measures outperform peer review in accuracy.

Predictive power consistent across disciplines and time windows.

Abstract

In this work, we try to answer the question of which method, peer review vs bibliometrics, better predicts the future overall scholarly impact of scientific publications. We measure the agreement between peer review evaluations of Web of Science indexed publications submitted to the first Italian research assessment exercise and long-term citations of the same publications. We do the same for an early citation-based indicator. We find that the latter shows stronger predictive power, i.e., it more reliably predicts late citations in all the disciplinary areas examined, and for any citation time window starting one year after publication.