The dynamically changing publication universe as a reference point in national impact evaluation: A counterfactual case study on the Chinese publication growth

TL;DR

This paper examines how structural changes in the publication environment, like China's rapid growth, influence impact metrics and emphasizes the need to account for such changes for unbiased national performance evaluation.

Contribution

It introduces a counterfactual approach to quantify environmental effects on impact metrics amid structural shifts in publication data.

Findings

Chinese publication growth benefits many countries' impact metrics

Longer reference lists influence impact calculations

Non-uniform citation distribution affects impact evaluations

Abstract

National bibliometric performance is commonly measured via relative impact indicators which appraise absolute national values through a global environment. Consequenty the resulting impact values mirror changes in the national performance as well as in its embedding. In order to assess the importance of the environment in this ratio, we analyse the increase in Chinese publications as an example for a structural change altering the whole database. Via a counterfactual comparison we quantify how Chinese publications benefit a large set of countries on their impact values, identify explanatory factors and describe the underelying mechanism due to longer reference lists and a non-uniform citation distribution among recipient countries. We argue that such structural changes in the environment have to be taken into account for an unbiased measurement of national bibliometric performance.