Through the Fairness Lens: Experimental Analysis and Evaluation of Entity Matching

TL;DR

This paper conducts an extensive experimental analysis of entity matching techniques to evaluate their fairness, revealing potential biases related to demographic overrepresentation and name similarity, and highlighting effective fairness measures.

Contribution

It is the first comprehensive study to evaluate the fairness of entity matching algorithms using social datasets and various fairness metrics.

Findings

Fairness issues arise when demographic groups are overrepresented.

Name similarity varies across groups, affecting matching fairness.

PPV parity and TPR parity are effective in detecting EM unfairness.

Abstract

Entity matching (EM) is a challenging problem studied by different communities for over half a century. Algorithmic fairness has also become a timely topic to address machine bias and its societal impacts. Despite extensive research on these two topics, little attention has been paid to the fairness of entity matching. Towards addressing this gap, we perform an extensive experimental evaluation of a variety of EM techniques in this paper. We generated two social datasets from publicly available datasets for the purpose of auditing EM through the lens of fairness. Our findings underscore potential unfairness under two common conditions in real-world societies: (i) when some demographic groups are overrepresented, and (ii) when names are more similar in some groups compared to others. Among our many findings, it is noteworthy to mention that while various fairness definitions are valuable for different settings, due to EM's class imbalance nature, measures such as positive predictive value parity and true positive rate parity are, in general, more capable of revealing EM unfairness.