Towards an Approach for Evaluating the Impact of AI Standards The use case of entity resolution

TL;DR

This paper explores how to evaluate AI standards specifically for entity resolution, emphasizing the importance of high-quality data for AI applications and proposing an approach to assess the impact of standards in this context.

Contribution

It introduces a framework for evaluating the impact of AI standards on entity resolution, focusing on data quality and the use case of learning employment records.

Findings

Standards can significantly improve entity resolution quality.

AI-based methods are essential for scalable, accurate entity matching.

Evaluation frameworks help measure the effectiveness of standards in real-world scenarios.

Abstract

This paper is intended to provide an overview of how the evaluation of standards could be applied to entity resolution, or record linkage. Data quality is of critical importance for many AI applications, and the quality of data, particularly on individuals and businesses, depends critically, in turn, on the quality of the match of entities across different files. Getting entity resolution right is important, because high quality data on entities like people or organization are essential to many AI systems; creating high quality data increasingly requires correctly classifying information that comes from different sources as generated by the same entity. But it is also very difficult because data on the same entity that are acquired from different sources are often inconsistent and have to be carefully reconciled. The use of AI, in the form of machine learning methods, is becoming increasingly important because other approaches are less applicable for modern needs. In particular, manual methods to link data are too costly and slow to scale, and probabilistic methods are inappropriate in the increasingly frequent cases where unique identifiers are not available. The particular focus is on Learning Employment Records, which is a high profile example of the value of entity resolution