Testing bibliometric indicators by their prediction of scientists promotions

TL;DR

This study evaluates various bibliometric indicators' effectiveness in predicting scientists' promotions across multiple disciplines, revealing that no single metric is universally best and highlighting the limitations of common indicators like citations.

Contribution

It introduces a robust method to analyze bibliometric indicators' predictive power for promotions across diverse scientific fields, challenging previous assumptions about indicator effectiveness.

Findings

Normalized h index varies with age for same productivity.

Hirsch index h is the best predictor among tested metrics.

No indicator predicts more than half of promotions accurately.

Abstract

We have developed a method to obtain robust quantitative bibliometric indicators for several thousand scientists. This allows us to study the dependence of bibliometric indicators (such as number of publications, number of citations, Hirsch index...) on the age, position, etc. of CNRS scientists. Our data suggests that the normalized h index (h divided by the career length) is not constant for scientists with the same productivity but differents ages. We also compare the predictions of several bibliometric indicators on the promotions of about 600 CNRS researchers. Contrary to previous publications, our study encompasses most disciplines, and shows that no single indicator is the best predictor for all disciplines. Overall, however, the Hirsch index h provides the least bad correlations, followed by the number of papers published. It is important to realize however that even h is able to recover only half of the actual promotions. The number of citations or the mean number of citations per paper are definitely not good predictors of promotion.