Testing the fairness of citation indicators for comparison across scientific domains: the case of fractional citation counts

TL;DR

This paper introduces a statistical test to evaluate the fairness of citation indicators across scientific disciplines, demonstrating that fractional citation counts do not effectively eliminate citation biases compared to other methods.

Contribution

It presents a novel statistical method for assessing the fairness of citation indicators and applies it to test the effectiveness of fractional citation counts in reducing disciplinary biases.

Findings

Fractional citation counts do not remove citation biases effectively.

Rescaling citation counts with average values performs better.

The proposed method is simple and adaptable to various scenarios.

Abstract

Citation numbers are extensively used for assessing the quality of scientific research. The use of raw citation counts is generally misleading, especially when applied to cross-disciplinary comparisons, since the average number of citations received is strongly dependent on the scientific discipline of reference of the paper. Measuring and eliminating biases in citation patterns is crucial for a fair use of citation numbers. Several numerical indicators have been introduced with this aim, but so far a specific statistical test for estimating the fairness of these numerical indicators has not been developed. Here we present a statistical method aimed at estimating the effectiveness of numerical indicators in the suppression of citation biases. The method is simple to implement and can be easily generalized to various scenarios. As a practical example we test, in a controlled case, the fairness of fractional citation count, which has been recently proposed as a tool for cross-discipline comparison. We show that this indicator is not able to remove biases in citation patterns and performs much worse than the rescaling of citation counts with average values.