Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus

TL;DR

This study systematically evaluates the accuracy of Web of Science and Scopus journal classification systems using citation-based criteria, revealing Web of Science's superior performance in classification accuracy.

Contribution

It introduces citation-based criteria to assess and compare the accuracy of journal classification systems in major bibliographic databases.

Findings

Web of Science has more accurate journal classifications than Scopus.

Citation-based criteria effectively identify questionable journal assignments.

The approach is validated within the Library and Information Science field.

Abstract

Journal classification systems play an important role in bibliometric analyses. The two most important bibliographic databases, Web of Science and Scopus, each provide a journal classification system. However, no study has systematically investigated the accuracy of these classification systems. To examine and compare the accuracy of journal classification systems, we define two criteria on the basis of direct citation relations between journals and categories. We use Criterion I to select journals that have weak connections with their assigned categories, and we use Criterion II to identify journals that are not assigned to categories with which they have strong connections. If a journal satisfies either of the two criteria, we conclude that its assignment to categories may be questionable. Accordingly, we identify all journals with questionable classifications in Web of Science and Scopus. Furthermore, we perform a more in-depth analysis for the field of Library and Information Science to assess whether our proposed criteria are appropriate and whether they yield meaningful results. It turns out that according to our citation-based criteria Web of Science performs significantly better than Scopus in terms of the accuracy of its journal classification system.