Benchmarking Declarative Approximate Selection Predicates

TL;DR

This paper introduces new similarity predicates for data quality tasks, provides their declarative SQL implementations, and compares their performance and accuracy in data cleaning applications.

Contribution

It proposes novel similarity predicates based on probabilistic information retrieval and offers a comprehensive benchmarking of various predicates.

Findings

New similarity predicates with declarative SQL implementations

Performance and accuracy comparison of multiple predicates

Insights into the suitability of predicates for data cleaning

Abstract

Declarative data quality has been an active research topic. The fundamental principle behind a declarative approach to data quality is the use of declarative statements to realize data quality primitives on top of any relational data source. A primary advantage of such an approach is the ease of use and integration with existing applications. Several similarity predicates have been proposed in the past for common quality primitives (approximate selections, joins, etc.) and have been fully expressed using declarative SQL statements. In this thesis, new similarity predicates are proposed along with their declarative realization, based on notions of probabilistic information retrieval. Then, full declarative specifications of previously proposed similarity predicates in the literature are presented, grouped into classes according to their primary characteristics. Finally, a thorough performance and accuracy study comparing a large number of similarity predicates for data cleaning operations is performed.