Deep Impact: Unintended consequences of journal rank

TL;DR

This paper reviews recent data showing that using journal rank as a measure of scientific impact leads to negative consequences, and advocates for replacing journals with a library-based communication system utilizing modern technology.

Contribution

It provides comprehensive data analysis on the detrimental effects of journal rank and proposes a transition to a library-based scholarly communication system.

Findings

Journal rank correlates poorly with scientific quality.

Using journal rank encourages undesirable research behaviors.

A library-based system could improve scientific communication.

Abstract

Most researchers acknowledge an intrinsic hierarchy in the scholarly journals ('journal rank') that they submit their work to, and adjust not only their submission but also their reading strategies accordingly. On the other hand, much has been written about the negative effects of institutionalizing journal rank as an impact measure. So far, contributions to the debate concerning the limitations of journal rank as a scientific impact assessment tool have either lacked data, or relied on only a few studies. In this review, we present the most recent and pertinent data on the consequences of our current scholarly communication system with respect to various measures of scientific quality (such as utility/citations, methodological soundness, expert ratings or retractions). These data corroborate previous hypotheses: using journal rank as an assessment tool is bad scientific practice. Moreover, the data lead us to argue that any journal rank (not only the currently-favored Impact Factor) would have this negative impact. Therefore, we suggest that abandoning journals altogether, in favor of a library-based scholarly communication system, will ultimately be necessary. This new system will use modern information technology to vastly improve the filter, sort and discovery functions of the current journal system.