The coverage of Microsoft Academic: Analyzing the publication output of a university

TL;DR

This study evaluates Microsoft Academic's coverage of university publications, comparing it with Scopus and WoS, highlighting strengths in document type coverage and identifying biases in humanities and open-access publications.

Contribution

First detailed assessment of Microsoft Academic's coverage and data quality compared to established bibliometric databases, including a publicly available metadata retrieval script.

Findings

MA covers more document types than Scopus and WoS.

MA has high citation count correlation with benchmark databases.

Coverage is favorable in sciences but biased in humanities and non-English works.

Abstract

This is the first detailed study on the coverage of Microsoft Academic (MA). Based on the complete and verified publication list of a university, the coverage of MA was assessed and compared with two benchmark databases, Scopus and Web of Science (WoS), on the level of individual publications. Citation counts were analyzed, and issues related to data retrieval and data quality were examined. A Perl script was written to retrieve metadata from MA based on publication titles. The script is freely available on GitHub. We find that MA covers journal articles, working papers, and conference items to a substantial extent and indexes more document types than the benchmark databases (e.g., working papers, dissertations). MA clearly surpasses Scopus and WoS in covering book-related document types and conference items but falls slightly behind Scopus in journal articles. The coverage of MA is favorable for evaluative bibliometrics in most research fields, including economics/business, computer/information sciences, and mathematics. However, MA shows biases similar to Scopus and WoS with regard to the coverage of the humanities, non-English publications, and open-access publications. Rank correlations of citation counts are high between MA and the benchmark databases. We find that the publication year is correct for 89.5% of all publications and the number of authors is correct for 95.1% of the journal articles. Given the fast and ongoing development of MA, we conclude that MA is on the verge of becoming a bibliometric superpower. However, comprehensive studies on the quality of MA metadata are still lacking.