Jointly Identifying and Fixing Inconsistent Readings from Information Extraction Systems

TL;DR

KGCleaner is a novel framework that jointly identifies and repairs inconsistent or incorrect information extracted from text, significantly improving the accuracy and credibility of knowledge graph data.

Contribution

It introduces the first multi-task model that simultaneously assesses credibility and repairs extracted facts, demonstrating substantial performance gains on large datasets.

Findings

Shallow neural networks improve credibility classification by 30 F1 points on Wikidata.

Significant performance gains in fact repair, more than doubling previous results.

Effective on large-scale datasets like Wikidata and TAC KBP.

Abstract

KGCleaner is a framework to identify and correct errors in data produced and delivered by an information extraction system. These tasks have been understudied and KGCleaner is the first to address both. We introduce a multi-task model that jointly learns to predict if an extracted relation is credible and repair it if not. We evaluate our approach and other models as instance of our framework on two collections: a Wikidata corpus of nearly 700K facts and 5M fact-relevant sentences and a collection of 30K facts from the 2015 TAC Knowledge Base Population task. For credibility classification, parameter efficient simple shallow neural network can achieve an absolute performance gain of 30 $F_1$ points on Wikidata and comparable performance on TAC. For the repair task, significant performance (at more than twice) gain can be obtained depending on the nature of the dataset and the models.