A systematic empirical comparison of different approaches for normalizing citation impact indicators

TL;DR

This paper empirically compares various normalization methods for citation impact indicators, finding that source normalization approaches generally outperform traditional classification-based methods in ensuring fair cross-field and cross-year comparisons.

Contribution

It provides a large-scale empirical evaluation of normalization techniques, especially highlighting the effectiveness of source normalization over classification-based methods using algorithmically constructed systems.

Findings

Source normalization generally outperforms classification-based normalization.

Fractional citation counting does not perform well.

Algorithmically constructed classification systems are used for evaluation.

Abstract

We address the question how citation-based bibliometric indicators can best be normalized to ensure fair comparisons between publications from different scientific fields and different years. In a systematic large-scale empirical analysis, we compare a traditional normalization approach based on a field classification system with three source normalization approaches. We pay special attention to the selection of the publications included in the analysis. Publications in national scientific journals, popular scientific magazines, and trade magazines are not included. Unlike earlier studies, we use algorithmically constructed classification systems to evaluate the different normalization approaches. Our analysis shows that a source normalization approach based on the recently introduced idea of fractional citation counting does not perform well. Two other source normalization approaches generally outperform the classification-system-based normalization approach that we study. Our analysis therefore offers considerable support for the use of source-normalized bibliometric indicators.