How large is the error effect when summing or averaging nonlinear field normalization citation counts at the paper level?

TL;DR

This study investigates the magnitude of errors introduced when summing or averaging nonlinear field-normalized citation counts, revealing that the errors are generally small but influenced by sample heterogeneity.

Contribution

The paper provides an empirical assessment of the error effects caused by nonlinear normalization methods when aggregating citation counts at the paper level.

Findings

Errors from nonlinear normalization are relatively small.

Sample heterogeneity significantly affects the error magnitude.

Linear normalization methods are more appropriate for aggregation.

Abstract

Summing or averaging nonlinearly field-normalized citation counts is a common but methodologically problematic practice, as it violates mathematical principles. The issue originates from the nonlinear transformation, which disrupts the equal-interval property of the data. Such unequal data do not satisfy the necessary conditions for summation. In our study, we normalized citation counts of papers from all sample universities using six linear and nonlinear methods, and then computed the total and average scores for each university under each method. By benchmarking against raw citations and linear normalized scores, we explore how large the error effect is from summing or averaging the nonlinear field normalized citation counts. Our empirical results indicate that the error exists but is relatively small. We further found that the magnitude of the error is significantly influenced by whether the sample publications are homogeneous or heterogeneous. This study has significant implications for whether the results obtained through nonlinear methods on a single level can be directly summed or averaged when calculating the overall impact of a research unit.