Evaluation of the citation matching algorithms of CWTS and iFQ in comparison to Web of Science

TL;DR

This study evaluates and compares the citation matching algorithms of CWTS, iFQ, and Web of Science, highlighting their accuracy and limitations in handling reference inaccuracies in bibliometric research.

Contribution

It provides a quantitative and qualitative comparison of CWTS and iFQ algorithms against WoS, revealing CWTS's superior performance in citation matching accuracy.

Findings

CWTS algorithm performs best among the three.

WoS has high overall performance but struggles with inaccurate references.

Inaccuracies in source articles affect citation matching quality.

Abstract

The results of bibliometric studies provided by bibliometric research groups, e.g. the Centre for Science and Technology Studies (CWTS) and the Institute for Research Information and Quality Assurance (iFQ), are often used in the process of research assessment. Their databases use Web of Science (WoS) citation data, which they match according to their own matching algorithms - in the case of CWTS for standard usage in their studies and in the case of iFQ on an experimental basis. Since the problem of non-matched citations in WoS persists because of inaccuracies in the references or inaccuracies introduced in the data extraction process, it is important to ascertain how well these inaccuracies are rectified in these citation matching algorithms. This paper evaluates the algorithms of CWTS and iFQ in comparison to WoS in a quantitative and a qualitative analysis. The analysis builds upon the methodology and the manually verified corpus of a previous study. The algorithm of CWTS performs best, closely followed by that of iFQ. The WoS algorithm still performs quite well (F1 score: 96.41 percent), but shows deficits in matching references containing inaccuracies. An additional problem is posed by incorrectly provided cited reference information in source articles by WoS.