TransClean: Finding False Positives in Multi-Source Entity Matching under Real-World Conditions via Transitive Consistency

TL;DR

TransClean is a novel method that detects false positives in multi-source entity matching by leveraging transitive consistency, improving matching accuracy without manual labeling under real-world noisy conditions.

Contribution

It introduces a transitive consistency-based approach for identifying false positives in multi-source entity matching, operating efficiently with limited manual labels and handling distributional shifts.

Findings

TransClean improves F1 score by an average of 24.42 points.

It effectively detects false positives across various datasets.

The method enhances existing matching models without retraining.

Abstract

We present TransClean, a method for detecting false positive predictions of entity matching algorithms under real-world conditions characterized by large-scale, noisy, and unlabeled multi-source datasets that undergo distributional shifts. TransClean is explicitly designed to operate with multiple data sources in an efficient, robust and fast manner while accounting for edge cases and requiring limited manual labeling. TransClean leverages the Transitive Consistency of a matching, a measure of the consistency of a pairwise matching model f_theta on the matching it produces G_f_theta, based both on its predictions on directly evaluated record pairs and its predictions on implied record pairs. TransClean iteratively modifies a matching through gradually removing false positive matches while removing as few true positive matches as possible. In each of these steps, the estimation of the Transitive Consistency is exclusively done through model evaluations and produces quantities that can be used as proxies of the amounts of true and false positives in the matching while not requiring any manual labeling, producing an estimate of the quality of the matching and indicating which record groups are likely to contain false positives. In our experiments, we compare combining TransClean with a naively trained pairwise matching model (DistilBERT) and with a state-of-the-art end-to-end matching method (CLER) and illustrate the flexibility of TransClean in being able to detect most of the false positives of either setup across a variety of datasets. Our experiments show that TransClean induces an average +24.42 F1 score improvement for entity matching in a multi-source setting when compared to traditional pair-wise matching algorithms.