Typing Errors in Factual Knowledge Graphs: Severity and Possible Ways Out

TL;DR

This paper investigates the high error rates in factual knowledge graphs like DBpedia and Wikidata, proposing an active detection algorithm and discussing various error correction paradigms to improve their quality.

Contribution

It introduces an active typing error detection method and compares different paradigms for handling errors in factual knowledge graphs.

Findings

Typing error rate is 27% for coarse types and up to 73% for fine types.

Proposed an active detection algorithm utilizing gold and noisy labels.

Provided guidelines and resources for improving and using noisy KGs.

Abstract

Factual knowledge graphs (KGs) such as DBpedia and Wikidata have served as part of various downstream tasks and are also widely adopted by artificial intelligence research communities as benchmark datasets. However, we found these KGs to be surprisingly noisy. In this study, we question the quality of these KGs, where the typing error rate is estimated to be 27% for coarse-grained types on average, and even 73% for certain fine-grained types. In pursuit of solutions, we propose an active typing error detection algorithm that maximizes the utilization of both gold and noisy labels. We also comprehensively discuss and compare unsupervised, semi-supervised, and supervised paradigms to deal with typing errors in factual KGs. The outcomes of this study provide guidelines for researchers to use noisy factual KGs. To help practitioners deploy the techniques and conduct further research, we published our code and data.