Complexity and Efficient Algorithms for Data Inconsistency Evaluating and Repairing

TL;DR

This paper investigates the complexity of data inconsistency evaluation and repair, providing new bounds, approximation algorithms, and a fast estimator for optimal repair size, enhancing data quality management techniques.

Contribution

It improves the understanding of the complexity of optimal subset repair, introduces new approximation algorithms, and develops a sublinear estimator for repair size in data inconsistency evaluation.

Findings

Optimal subset repair is NP-hard and APX-complete.

New approximation algorithms with specific ratios are proposed.

A sublinear estimator for repair size achieves high-probability accuracy.

Abstract

Data inconsistency evaluating and repairing are major concerns in data quality management. As the basic computing task, optimal subset repair is not only applied for cost estimation during the progress of database repairing, but also directly used to derive the evaluation of database inconsistency. Computing an optimal subset repair is to find a minimum tuple set from an inconsistent database whose remove results in a consistent subset left. Tight bound on the complexity and efficient algorithms are still unknown. In this paper, we improve the existing complexity and algorithmic results, together with a fast estimation on the size of optimal subset repair. We first strengthen the dichotomy for optimal subset repair computation problem, we show that it is not only APXcomplete, but also NPhard to approximate an optimal subset repair with a factor better than $17/16$ for most cases. We second show a $(2-0.5^{\tiny\sigma-1})$-approximation whenever given $\sigma$ functional dependencies, and a $(2-\eta_k+\frac{\eta_k}{k})$-approximation when an $\eta_k$-portion of tuples have the $k$-quasi-Tur$\acute{\text{a}}$n property for some $k>1$. We finally show a sublinear estimator on the size of optimal \textit{S}-repair for subset queries, it outputs an estimation of a ratio $2n+\epsilon n$ with a high probability, thus deriving an estimation of FD-inconsistency degree of a ratio $2+\epsilon$. To support a variety of subset queries for FD-inconsistency evaluation, we unify them as the $\subseteq$-oracle which can answer membership-query, and return $p$ tuples uniformly sampled whenever given a number $p$. Experiments are conducted on range queries as an implementation of $\subseteq$-oracle, and results show the efficiency of our FD-inconsistency degree estimator.