Universities through the Eyes of Bibliographic Databases: A Retroactive Growth Comparison of Google Scholar, Scopus and Web of Science

TL;DR

This study evaluates the effectiveness of Google Scholar's URL-based method for measuring university research output, finding it unreliable for total publication counts but moderately correlated with traditional databases, influenced by open access practices.

Contribution

It provides an empirical comparison of bibliographic databases, highlighting limitations of Google Scholar's URL-based productivity estimates at the institutional level.

Findings

Retroactive growth in GS is unpredictable and varies by university.

Moderate correlation exists between GS, WoS, and Scopus productivity measures.

Only 16% of articles are indexed in official university websites, affecting productivity estimates.

Abstract

The purpose of this study is to ascertain the suitability of GS's url-based method as a valid approximation of universities' academic output measures, taking into account three aspects (retroactive growth, correlation, and coverage). To do this, a set of 100 Turkish universities were selected as a case study. The productivity in Web of Science (WoS), Scopus and GS (2000 to 2013) were captured in two different measurement iterations (2014 and 2018). In addition, a total of 18,174 documents published by a subset of 14 research-focused universities were retrieved from WoS, verifying their presence in GS within the official university web domain. Findings suggest that the retroactive growth in GS is unpredictable and dependent on each university, making this parameter hard to evaluate at the institutional level. Otherwise, the correlation of productivity between GS (url-based method) and WoS and Scopus (selected sources) is moderately positive, even though it varies depending on the university, the year of publication, and the year of measurement. Finally, only 16% out of 18,174 articles analyzed were indexed in the official university website, although up to 84% were indexed in other GS sources. This work proves that the url-based method to calculate institutional productivity in GS is not a good proxy for the total number of publications indexed in WoS and Scopus, at least in the national context analyzed. However, the main reason is not directly related to the operation of GS, but with a lack of universities' commitment to open access.