More Precise Methods for National Research Citation Impact Comparisons

TL;DR

This paper introduces two new statistical methods, based on linear modeling and geometric means, to improve the precision of national research citation impact comparisons, especially for small samples.

Contribution

It proposes and evaluates two novel methods for more accurate national citation impact analysis, with geometric means being recommended for smaller samples.

Findings

Geometric means provide the most precise impact estimates for small samples.

Regression-based method distinguishes national contributions in collaborative papers.

Geometric mean method is recommended for smaller sample sizes.

Abstract

Governments sometimes need to analyse sets of research papers within a field in order to monitor progress, assess the effect of recent policy changes, or identify areas of excellence. They may compare the average citation impacts of the papers by dividing them by the world average for the field and year. Since citation data is highly skewed, however, simple averages may be too imprecise to robustly identify differences within, rather than across, fields. In response, this article introduces two new methods to identify national differences in average citation impact, one based on linear modelling for normalised data and the other using the geometric mean. Results from a sample of 26 Scopus fields between 2009-2015 show that geometric means are the most precise and so are recommended for smaller sample sizes, such as for individual fields. The regression method has the advantage of distinguishing between national contributions to internationally collaborative articles, but has substantially wider confidence intervals than the geometric mean, undermining its value for any except the largest sample sizes.