Confirmation Bias and the Open Access Advantage: Some Methodological Suggestions for the Davis Citation Study

TL;DR

This paper critiques Davis (2008)'s analysis of open access citation advantages, highlighting methodological issues and proposing improvements to better understand the true effects and causes of OA citation benefits.

Contribution

It identifies key methodological shortcomings in previous OA citation studies and suggests comprehensive analytical adjustments to accurately assess OA's impact.

Findings

OA citation advantage is small and possibly declining.

Self-archiving may contribute significantly to OA benefits.

Paid OA might not be cost-effective for increasing citations.

Abstract

Davis (2008) analyzes citations from 2004-2007 in 11 biomedical journals. 15% of authors paid to make them Open Access (OA). The outcome is a significant OA citation Advantage, but a small one (21%). The author infers that the OA advantage has been shrinking yearly, but the data suggest the opposite. Further analyses are necessary: (1) Not just author-choice (paid) OA but Free OA self-archiving needs to be taken into account rather than being counted as non-OA. (2) proportion of OA articles per journal per year needs to be reported and taken into account. (3) The Journal Impact Factor and the relation between the size of the OA Advantage article 'citation-bracket' need to be taken into account. (4) The sample-size for the highest-impact, largest-sample journal analyzed, PNAS, is restricted and excluded from some of the analyses. The full PNAS dataset is needed. (5) The interaction between OA and time, 2004-2007, is based on retrospective data from a June 2008 total cumulative citation count. The dates of both the cited articles and the citing articles need to be taken into account. The author proposes that author self-selection bias for is the primary cause of the observed OA Advantage, but this study does not test this or of any of the other potential causal factors. The author suggests that paid OA is not worth the cost, per extra citation. But with OA self-archiving both the OA and the extra citations are free.