Integrated Impact Indicators (I3) compared with Impact Factors (IFs): An alternative research design with policy implications

TL;DR

This paper introduces the Integrated Impact Indicator (I3), an alternative to impact factors that sums normalized citation curves using percentiles, providing a more accurate and decomposable measure of research impact with policy implications.

Contribution

It proposes and validates the I3 metric, which integrates citation curves normalized by percentiles, offering a more comprehensive and decomposable impact measure than traditional impact factors.

Findings

I3 correlates more accurately with actual impact than IFs.

I3 allows impact assessment at various levels, including journals and nations.

The study demonstrates the limitations of impact factors and the advantages of I3.

Abstract

In bibliometrics, the association of "impact" with central-tendency statistics is mistaken. Impacts add up, and citation curves should therefore be integrated instead of averaged. For example, the journals MIS Quarterly and JASIST differ by a factor of two in terms of their respective impact factors (IF), but the journal with the lower IF has the higher impact. Using percentile ranks (e.g., top-1%, top-10%, etc.), an integrated impact indicator (I3) can be based on integration of the citation curves, but after normalization of the citation curves to the same scale. The results across document sets can be compared as percentages of the total impact of a reference set. Total number of citations, however, should not be used instead because the shape of the citation curves is then not appreciated. I3 can be applied to any document set and any citation window. The results of the integration (summation) are fully decomposable in terms of journals or instititutional units such as nations, universities, etc., because percentile ranks are determined at the paper level. In this study, we first compare I3 with IFs for the journals in two ISI Subject Categories ("Information Science & Library Science" and "Multidisciplinary Sciences"). The LIS set is additionally decomposed in terms of nations. Policy implications of this possible paradigm shift in citation impact analysis are specified.