Inferring the causal effect of journals on citations

TL;DR

This paper investigates whether high-impact journals cause higher citation counts or simply select more citable articles, revealing that they both select and augment citations, which impacts research evaluation practices.

Contribution

It introduces a causal inference approach comparing preprints and published articles to disentangle journal impact from article quality effects.

Findings

High-impact journals select articles that attract more citations.

High-impact journals increase the citation rate of published articles.

Removing impact factors from evaluation does not eliminate journal influence.

Abstract

Articles in high-impact journals are, on average, more frequently cited. But are they cited more often because those articles are somehow more "citable"? Or are they cited more often simply because they are published in a high-impact journal? Although some evidence suggests the latter, the causal relationship is not clear. We here compare citations of preprints to citations of the published version to uncover the causal mechanism. We build on an earlier model of citation dynamics to infer the causal effect of journals on citations. We find that high-impact journals select articles that tend to attract more citations. At the same time, we find that high-impact journals augment the citation rate of published articles. Our results yield a deeper understanding of the role of journals in the research system. The use of journal metrics in research evaluation has been increasingly criticized in recent years and article-level citations are sometimes suggested as an alternative. Our results show that removing impact factors from evaluation does not negate the influence of journals. This insight has important implications for changing practices of research evaluation.