Enabling Quality Control for Entity Resolution: A Human and Machine Cooperation Framework

TL;DR

This paper introduces HUMO, a framework that combines human and machine efforts for entity resolution, providing flexible quality control and reducing human effort while maintaining high accuracy.

Contribution

The paper proposes a novel HUMO framework with three optimization strategies for balancing human cost and quality guarantees in entity resolution.

Findings

HUMO achieves high-quality results with lower human effort.

It outperforms existing methods in quality control.

Extensive experiments validate its effectiveness.

Abstract

Even though many machine algorithms have been proposed for entity resolution, it remains very challenging to find a solution with quality guarantees. In this paper, we propose a novel HUman and Machine cOoperation (HUMO) framework for entity resolution (ER), which divides an ER workload between the machine and the human. HUMO enables a mechanism for quality control that can flexibly enforce both precision and recall levels. We introduce the optimization problem of HUMO, minimizing human cost given a quality requirement, and then present three optimization approaches: a conservative baseline one purely based on the monotonicity assumption of precision, a more aggressive one based on sampling and a hybrid one that can take advantage of the strengths of both previous approaches. Finally, we demonstrate by extensive experiments on real and synthetic datasets that HUMO can achieve high-quality results with reasonable return on investment (ROI) in terms of human cost, and it performs considerably better than the state-of-the-art alternatives in quality control.