Fractional counting of citations in research evaluation: An option for cross- and interdisciplinary assessments

TL;DR

This paper proposes a fractional citation counting method based on reference list lengths in citing papers to improve cross- and interdisciplinary research evaluations, allowing fairer comparisons across fields.

Contribution

It introduces a novel normalization technique for citation counts that accounts for field-specific citation behaviors at the paper level.

Findings

The method enables statistically significant comparisons among research units.

Application to Tsinghua University departments demonstrated improved ranking accuracy.

Most departments showed no significant difference in impact after normalization.

Abstract

In the case of the scientometric evaluation of multi- or interdisciplinary units one risks to compare apples with oranges: each paper has to assessed in comparison to an appropriate reference set. We suggest that the set of citing papers first can be considered as the relevant representation of the field of impact. In order to normalize for differences in citation behavior among fields, citations can be fractionally counted proportionately to the length of the reference lists in the citing papers. This new method enables us to compare among units with different disciplinary affiliations at the paper level and also to assess the statistical significance of differences among sets. Twenty-seven departments of the Tsinghua University in Beijing are thus compared. Among them, the Department of Chinese Language and Linguistics is upgraded from the 19th to the second position in the ranking. The overall impact of 19 of the 27 departments is not significantly different at the 5% level when thus normalized for different citation potentials.