On Leveraging Large Language Models for Enhancing Entity Resolution: A Cost-efficient Approach

TL;DR

This paper presents a cost-efficient framework leveraging Large Language Models for entity resolution, reducing API costs and improving accuracy through uncertainty reduction and strategic querying.

Contribution

It introduces an uncertainty reduction framework with algorithms for selective querying and error tolerance, enhancing LLM-based entity resolution at scale.

Findings

Significant cost reduction in LLM API usage.

Improved accuracy in entity resolution tasks.

Effective handling of LLM errors and dynamic partition adjustment.

Abstract

Entity resolution, the task of identifying and merging records that refer to the same real-world entity, is crucial in sectors like e-commerce, healthcare, and law enforcement. Large Language Models (LLMs) introduce an innovative approach to this task, capitalizing on their advanced linguistic capabilities and a ``pay-as-you-go'' model that provides significant advantages to those without extensive data science expertise. However, current LLMs are costly due to per-API request billing. Existing methods often either lack quality or become prohibitively expensive at scale. To address these problems, we propose an uncertainty reduction framework using LLMs to improve entity resolution results. We first initialize possible partitions of the entity cluster, refer to the same entity, and define the uncertainty of the result. Then, we reduce the uncertainty by selecting a few valuable matching questions for LLM verification. Upon receiving the answers, we update the probability distribution of the possible partitions. To further reduce costs, we design an efficient algorithm to judiciously select the most valuable matching pairs to query. Additionally, we create error-tolerant techniques to handle LLM mistakes and a dynamic adjustment method to reach truly correct partitions. Experimental results show that our method is efficient and effective, offering promising applications in real-world tasks.