Does Open Access Really Increase Impact? A Large-Scale Randomized Analysis

TL;DR

This large-scale randomized study finds no citation advantage for fully open access journals and a diminishing advantage in hybrid journals over 2010-2020, possibly due to increased access via pirate sites.

Contribution

The paper introduces a control group methodology to accurately measure the open access citation advantage, revealing nuanced effects in different journal types and over time.

Findings

No citation advantage for fully OA journals, with a slight disadvantage.

Significant citation advantage in hybrid journals, decreasing over time.

The OACA in hybrid journals dropped from 70% to 9% between 2016 and 2020.

Abstract

The Open Access Citation Advantage (OACA) has been a major topic of discussion in the literature over the past twenty years. In this paper, we propose a method to constitute a control group to isolate the OACA effect. Thus, we compared citation impact (MNCS) of 2,458,378 publications in fully OA journals to that (weighted MNCS) of a control group of non-OA publications (\#10,310,842). Similarly, we did the same exercise for OA publications in hybrid journals (\#1,024,430) and their control group (\#11,533,001), over the period 2010-2020. The results showed that there is no OACA for publications in fully OA journals, and that there is rather a disadvantage. Conversely, the OACA seems to be a reality in hybrid journals, suggesting that a better accessibility in this context tends to improve the visibility of publications. The lack of OACA for publications in fully OA journals is to be expected, as a great proportion of OA journals are newly created and less attractive to high-impact senior researchers.Another striking result of this paper is the fall of the OACA from 2016. The citation advantage fell from 70% to 9% between 2016 and 2020 (for publications in hybrid journals). We wonder if this fall is linked to the increase in the notoriety of pirate sites (eg Sci-Hub) from 2016. In other words, the democratization of pirate sites instantly cancels the positive effect of OA publication insofar as the question of access to scientific content no longer arises.