Updatable Materialization of Approximate Constraints

TL;DR

This paper introduces an update-conscious PatchIndex structure using sharded bitmaps to efficiently maintain approximate constraints in dynamic data warehouses, enhancing query performance without costly recomputations.

Contribution

It extends PatchIndexes with a sharded bitmap design for efficient updates, enabling better support for dynamic data warehouse workloads.

Findings

PatchIndexes significantly improve query performance.

The sharded bitmap enables lightweight updates.

The approach avoids full table scans during updates.

Abstract

Modern big data applications integrate data from various sources. As a result, these datasets may not satisfy perfect constraints, leading to sparse schema information and non-optimal query performance. The existing approach of PatchIndexes enable the definition of approximate constraints and improve query performance by exploiting the materialized constraint information. As real world data warehouse workloads are often not limited to read-only queries, we enhance the PatchIndex structure towards an update-conscious design in this paper. Therefore, we present a sharded bitmap as the underlying data structure which offers efficient update operations, and describe approaches to maintain approximate constraints under updates, avoiding index recomputations and full table scans. In our evaluation, we prove that PatchIndexes significantly impact query performance while achieving lightweight update support.