Automatic Data Repair: Are We Ready to Deploy?

TL;DR

This paper systematically evaluates 12 data repair algorithms across various error scenarios and proposes a unified optimization strategy, providing practical guidelines and insights for deploying data repair in real-world applications.

Contribution

It offers a comprehensive comparison and taxonomy of data repair algorithms, introduces a novel evaluation metric, and presents a unified repair optimization strategy with practical deployment guidelines.

Findings

Data repair improves analysis performance regardless of error rate.

Pure clean data does not always yield optimal results.

The unified repair strategy significantly enhances existing methods.

Abstract

Data quality is paramount in today's data-driven world, especially in the era of generative AI. Dirty data with errors and inconsistencies usually leads to flawed insights, unreliable decision-making, and biased or low-quality outputs from generative models. The study of repairing erroneous data has gained significant importance. Existing data repair algorithms differ in information utilization, problem settings, and are tested in limited scenarios. In this paper, we initially compare and summarize these algorithms using a new guided information-based taxonomy. We then systematically conduct a comprehensive evaluation of 12 mainstream data repair algorithms under the settings of various data error rates, error types, and downstream analysis tasks, assessing their error reduction performance with a novel metric. Also, we develop an effective and unified repair optimization strategy that substantially benefits the state of the arts, as empirically confirmed. We demonstrate that, the pure clean data may not necessarily yield the best performance in data analysis tasks and data is always worth repairing regardless of error rate. Based on the found observations and insights, we provide some practical guidelines for 5 scenarios and 2 main data analysis tasks. We anticipate this paper enabling researchers and users to well understand and deploy data repair algorithms in practice. Finally, we outline research challenges and promising future directions in the data repair field.