Can the Journal Impact Factor Be Used as a Criterion for the Selection of Junior Researchers? A Large-Scale Empirical Study Based on ResearcherID Data

TL;DR

This large-scale empirical study investigates whether the normalized Journal Impact Factor can predict the future success of early-career researchers, finding it has some predictive power but should not be used as the sole criterion.

Contribution

The paper provides evidence on the predictive validity of normalized JIF for early-career researcher success, highlighting its limitations and suggesting additional criteria for assessment.

Findings

Normalized JIF can distinguish high-impact future publications.

JIF has low to medium effect sizes in predicting success.

Other factors should complement JIF in researcher evaluation.

Abstract

Early in researchers' careers, it is difficult to assess how good their work is or how important or influential the scholars will eventually be. Hence, funding agencies, academic departments, and others often use the Journal Impact Factor (JIF) of where the authors have published to assess their work and provide resources and rewards for future work. The use of JIFs in this way has been heavily criticized, however. Using a large data set with many thousands of publication profiles of individual researchers, this study tests the ability of the JIF (in its normalized variant) to identify, at the beginning of their careers, those candidates who will be successful in the long run. Instead of bare JIFs and citation counts, the metrics used here are standardized according to Web of Science subject categories and publication years. The results of the study indicate that the JIF (in its normalized variant) is able to discriminate between researchers who published papers later on with a citation impact above or below average in a field and publication year - not only in the short term, but also in the long term. However, the low to medium effect sizes of the results also indicate that the JIF (in its normalized variant) should not be used as the sole criterion for identifying later success: other criteria, such as the novelty and significance of the specific research, academic distinctions, and the reputation of previous institutions, should also be considered.