An Explorative Study on Document Type Assignment of Review Articles in Web of Science, Scopus and Journals' Website

TL;DR

This study evaluates the accuracy of review article classification in Web of Science, Scopus, and journal websites, revealing high precision but variable recall, especially for implicit reviews, and suggests improvements for better labeling practices.

Contribution

It provides a large-scale comparison of document type assignment accuracy across databases and identifies factors affecting correct classification of review articles.

Findings

WoS and Scopus have about 99% precision in classifying review articles.

Recall for review articles is around 80%, with lower accuracy for implicit reviews.

Differences exist between databases and journal series in classification accuracy.

Abstract

Accurately assigning the document type of review articles in citation index databases like Web of Science(WoS) and Scopus is important. This study aims to investigate the document type assignation of review articles in web of Science, Scopus and Journals' website in a large scale. 27,616 papers from 160 journals from 10 review journal series indexed in SCI are analyzed. The document types of these papers labeled on journals' website, and assigned by WoS and Scopus are retrieved and compared to determine the assigning accuracy and identify the possible reasons of wrongly assigning. For the document type labeled on the website, we further differentiate them into explicit review and implicit review based on whether the website directly indicating it is review or not. We find that WoS and Scopus performed similarly, with an average precision of about 99% and recall of about 80%. However, there were some differences between WoS and Scopus across different journal series and within the same journal series. The assigning accuracy of WoS and Scopus for implicit reviews dropped significantly. This study provides a reference for the accuracy of document type assigning of review articles in WoS and Scopus, and the identified pattern for assigning implicit reviews may be helpful to better labeling on website, WoS and Scopus.