Evaluating the Accuracy of the Labeling System in Web of Science for the Sustainable Development Goals

TL;DR

This study evaluates the Web of Science SDGs labeling system's accuracy by comparing it with the Bergon approach, revealing significant misclassifications and underrepresentation issues that highlight the need for ongoing improvements.

Contribution

It provides a systematic assessment of WoS SDGs labeling accuracy, identifying specific SDGs and topics with low relatedness and underrepresentation, informing future refinement efforts.

Findings

15% of topics show low relatedness to assigned SDGs

Higher misclassification rates in SDGs '11 Cities', '07 Energy', '13 Climate'

Underrepresentation of topics in SDGs '02 Hunger', '12 Consumption', '15 Land'

Abstract

Monitoring and fostering research aligned with the Sustainable Development Goals (SDGs) is crucial for formulating evidence-based policies, identifying best practices, and promoting global collaboration. The key step is developing a labeling system to map research publications to their related SDGs. The SDGs labeling system integrated in Web of Science (WoS), which assigns citation topics instead of individual publication to SDGs, has emerged as a promising tool.However we still lack of a comprehensive evaluation of the performance of WoS labeling system. By comparing with the Bergon approach, we systematically assessed the relatedness between citation topics and SDGs. Our analysis identified 15% of topics showing low relatedness to their assigned SDGs at a 1% threshold. Notably, SDGs such as '11 Cities', '07 Energy', and '13 Climate' exhibited higher percentages of low related topics. In addition, we revealed that certain topics are significantly underrepresented in their relevant SDGs, particularly for '02 Hunger', '12 Consumption', and '15 Land'. This study underscores the critical need for continual refinement and validation of SDGs labeling systems in WoS.