Assessing the accuracy of the h and g indexes for measuring researchers' productivity

TL;DR

This study evaluates the reliability of h- and g-indexes in ranking researchers by comparing them with a more comprehensive indicator, FSS, using Italian scientists' data from 2001-2005.

Contribution

It provides an empirical comparison of common bibliometric indexes against a novel FSS measure in national research assessments.

Findings

Significant rank shifts occur when using h- or g-index compared to FSS.

The FSS indicator offers a more nuanced assessment of individual research productivity.

Simple indexes may misrepresent researchers' true scientific impact.

Abstract

Bibliometric indicators are increasingly used in support of decisions for recruitment, career advancement, rewarding and selective funding for scientists. Given the importance of the applications, bibliometricians are obligated to carry out empirical testing of the robustness of the indicators, in simulations of real contexts. In this work we compare the results of national-scale research assessments at the individual level, based on three different indexes: the h-index, g-index and "fractional scientific strength", or FSS, an indicator previously proposed by the authors. For each index, we construct and compare rankings lists of all Italian academic researchers working in the hard sciences over the period 2001-2005. The analysis quantifies the shifts in ranks that occur when researchers' productivity rankings by simple indicators such as h- or g-index are compared with that by more accurate FSS.