Distinguishing articles in questionable and non-questionable journals using quantitative indicators associated with quality

TL;DR

This study compares quantitative indicators of article quality across questionable and reputable psychology journals, revealing differences but also highlighting quality concerns in mid-tier and indexed journals.

Contribution

It provides an empirical analysis of various quality indicators in articles from questionable, mid-tier, and indexed journals, highlighting differences and similarities.

Findings

QJ articles diverge from disciplinary standards on quality indicators

Mid-tier and WoS journals also show potential quality issues

Ethics reporting and statistical errors are common across journal types

Abstract

This study investigates the viability of distinguishing articles in questionable journals (QJs) from those in non-QJs on the basis of quantitative indicators typically associated with quality. Subsequently, I examine what can be deduced about the quality of articles in QJs based on the differences observed. I contrast the length of abstracts and full-texts, prevalence of spelling errors, text readability, number of references and citations, the size and internationality of the author team, the documentation of ethics and informed consent statements, and the presence erroneous decisions based on statistical errors in 1,714 articles from 31 QJs, 1,691 articles from 16 journals indexed in Web of Science (WoS), and 1,900 articles from 45 mid-tier journals, all in the field of psychology. The results suggest that QJ articles do diverge from the disciplinary standards set by peer-reviewed journals in psychology on quantitative indicators of quality that tend to reflect the effect of peer review and editorial processes. However, mid-tier and WoS journals are also affected by potential quality concerns, such as under-reporting of ethics and informed consent processes and the presence of errors in interpreting statistics. Further research is required to develop a comprehensive understanding of the quality of articles in QJs.