A new methodology for comparing Google Scholar and Scopus

TL;DR

This paper introduces a new methodology for comparing Google Scholar and Scopus, analyzing coverage, citation impact, indexing speed, and data quality across selected journals and fields, confirming some prior findings.

Contribution

It presents an exploratory, hypothesis-generating method for comparing citation indexes, focusing on coverage, impact, and data quality, with application to a limited journal set.

Findings

Google Scholar's citation counts are 1 to 4 times higher than Scopus, varying by field.

High correlation (R=0.8-0.9) between GS and Scopus citation counts at the article level.

Median Scopus indexing delay of two months compared to GS.

Abstract

A new methodology is proposed for comparing Google Scholar (GS) with other citation indexes. It focuses on the coverage and citation impact of sources, indexing speed, and data quality, including the effect of duplicate citation counts. The method compares GS with Elsevier's Scopus, and is applied to a limited set of articles published in 12 journals from six subject fields, so that its findings cannot be generalized to all journals or fields. The study is exploratory, and hypothesis generating rather than hypothesis-testing. It confirms findings on source coverage and citation impact obtained in earlier studies. The ratio of GS over Scopus citation varies across subject fields between 1.0 and 4.0, while Open Access journals in the sample show higher ratios than their non-OA counterparts. The linear correlation between GS and Scopus citation counts at the article level is high: Pearson's R is in the range of 0.8-0.9. A median Scopus indexing delay of two months compared to GS is largely though not exclusively due to missing cited references in articles in press in Scopus. The effect of double citation counts in GS due to multiple citations with identical or substantially similar meta-data occurs in less than 2 per cent of cases. Pros and cons of article-based and what is termed as concept-based citation indexes are discussed.