F1000 recommendations as a new data source for research evaluation: A comparison with citations

TL;DR

This study compares F1000 post-publication recommendations with traditional citation metrics in biomedical research, revealing a weak correlation and highlighting potential as an alternative research evaluation tool.

Contribution

It introduces F1000 recommendations as a new data source for research evaluation and compares its effectiveness with citation-based metrics.

Findings

2% of biomedical publications receive F1000 recommendations

Recommended papers average 1.30 recommendations

Over 90% of recommendations occur within six months of publication

Abstract

F1000 is a post-publication peer review service for biological and medical research. F1000 aims to recommend important publications in the biomedical literature, and from this perspective F1000 could be an interesting tool for research evaluation. By linking the complete database of F1000 recommendations to the Web of Science bibliographic database, we are able to make a comprehensive comparison between F1000 recommendations and citations. We find that about 2% of the publications in the biomedical literature receive at least one F1000 recommendation. Recommended publications on average receive 1.30 recommendations, and over 90% of the recommendations are given within half a year after a publication has appeared. There turns out to be a clear correlation between F1000 recommendations and citations. However, the correlation is relatively weak, at least weaker than the correlation between journal impact and citations. More research is needed to identify the main reasons for differences between recommendations and citations in assessing the impact of publications.