ActiveClean: Interactive Data Cleaning While Learning Convex Loss Models

TL;DR

ActiveClean is an interactive data cleaning method that incrementally updates convex loss models, prioritizing data likely to impact results, leading to more accurate models with less cleaning effort.

Contribution

It introduces ActiveClean, a progressive cleaning approach that guarantees model accuracy on partially cleaned data and efficiently prioritizes data cleaning based on model structure.

Findings

Improves model accuracy up to 2.5x with the same cleaning effort

Outperforms uniform sampling and active learning in real datasets

Supports convex loss models like linear regression and SVMs

Abstract

Data cleaning is often an important step to ensure that predictive models, such as regression and classification, are not affected by systematic errors such as inconsistent, out-of-date, or outlier data. Identifying dirty data is often a manual and iterative process, and can be challenging on large datasets. However, many data cleaning workflows can introduce subtle biases into the training processes due to violation of independence assumptions. We propose ActiveClean, a progressive cleaning approach where the model is updated incrementally instead of re-training and can guarantee accuracy on partially cleaned data. ActiveClean supports a popular class of models called convex loss models (e.g., linear regression and SVMs). ActiveClean also leverages the structure of a user's model to prioritize cleaning those records likely to affect the results. We evaluate ActiveClean on five real-world datasets UCI Adult, UCI EEG, MNIST, Dollars For Docs, and WorldBank with both real and synthetic errors. Our results suggest that our proposed optimizations can improve model accuracy by up-to 2.5x for the same amount of data cleaned. Furthermore for a fixed cleaning budget and on all real dirty datasets, ActiveClean returns more accurate models than uniform sampling and Active Learning.