Evaluating author name disambiguation for digital libraries: A case of DBLP

TL;DR

This study evaluates the accuracy of author name disambiguation in DBLP using multiple datasets, showing high overall performance but challenges with same-name authors, and compares it to other algorithms.

Contribution

It introduces a comprehensive evaluation of DBLP's disambiguation performance across diverse datasets, highlighting its strengths and weaknesses.

Findings

DBLP achieves pairwise precision, recall, and F1 scores around 0.90 or higher.

Disambiguation is less effective for authors with identical names.

DBLP's hybrid approach combines algorithmic and manual methods, contributing to its performance.

Abstract

Author name ambiguity in a digital library may affect the findings of research that mines authorship data of the library. This study evaluates author name disambiguation in DBLP, a widely used but insufficiently evaluated digital library for its disambiguation performance. In doing so, this study takes a triangulation approach that author name disambiguation for a digital library can be better evaluated when its performance is assessed on multiple labeled datasets with comparison to baselines. Tested on three types of labeled data containing 5,000 ~ 700K disambiguated names and 6M pairs of disambiguated names, DBLP is shown to assign author names quite accurately to distinct authors, resulting in pairwise precision, recall, and F1 measures around 0.90 or above overall. DBLP's author name disambiguation performs well even on large ambiguous name blocks but deficiently on distinguishing authors with the same names. When compared to other disambiguation algorithms, DBLP's disambiguation performance is quite competitive, possibly due to its hybrid disambiguation approach combining algorithmic disambiguation and manual error correction. A discussion follows on strengths and weaknesses of labeled datasets used in this study for future efforts to evaluate author name disambiguation on a digital library scale.