A logical set theory approach to journal subject classification analysis: intra-system irregularities and inter-system discrepancies in Web of Science and Scopus

TL;DR

This study analyzes the irregularities and discrepancies in journal subject classifications between Web of Science and Scopus using a logical set theory approach, revealing significant inconsistencies that impact research evaluation.

Contribution

It introduces a logical set theory-based framework to identify and quantify classification irregularities and discrepancies across major bibliometric systems.

Findings

Unusually sized categories identified in both systems

High overlap and incohesiveness between categories

Systematic differences in journal classification across systems

Abstract

Journal classification into subject categories is an important aspect in scholarly research evaluation as well as in bibliometric analysis. However, this classification is not standardized, resulting in several different journal subject classification systems. In this study, we adopt a logical set theory-based definition of irregularities within a given classification system and discrepancies between systems and investigate their prevalence in the two most widely used indexing services of Web of Science (WoS) and Scopus. In both systems, we identify unusually sized categories, high overlap and incohesiveness between categories. In addition, across the two systems, journals are systematically classified to a different number of categories and most categories in either system are not adequately represented in the other system. Our findings suggest that these irregularities and discrepancies are, in fact, non-anecdotal and thus cannot be easily disregarded. Consequently, potentially misguided and/or inconsistent outcomes may be encountered when relying on these subject classification systems.