A DBMS-independent approach for capturing provenance polynomials through query rewriting

TL;DR

This paper introduces a DBMS-independent query rewriting approach to capture provenance polynomials, supporting complex SQL operations and nested queries, with an implementation that outperforms existing systems in scalability and performance.

Contribution

It presents the first full implementation of semiring-based provenance polynomials extended with semimodule structures, supporting a wide range of SQL operations in a DBMS-independent manner.

Findings

Supports complex SQL queries including aggregations and nested queries.

Demonstrates improved performance and scalability over existing methods.

Provides a comprehensive implementation of theoretical provenance formalisms.

Abstract

In today's data-driven ecosystems, ensuring data integrity, traceability and accountability is important. Provenance polynomials constitute a powerful formalism for tracing the origin and the derivations made to produce database query results. Despite their theoretical expressiveness, current implementations have limitations in handling aggregations and nested queries, and some of them and tightly coupled to a single Database Management System (DBMS), hindering interoperability and broader applicability. This paper presents a query rewriting-based approach for annotating Structured Query Language (SQL) queries with provenance polynomials. The proposed methods are DBMS-independent and support Select-Projection-Join-Union-Aggregation (SPJUA) operations and nested queries, through recursive propagation of provenance annotations. This constitutes the first full implementation of semiring-based theory for provenance polynomials extended with semimodule structures. It also presents an experimental evaluation to assess the validity of the proposed methods and compare the performance against state-of-the-art systems using benchmark data and queries. The results indicate that our solution delivers a comprehensive implementation of the theoretical formalisms proposed in the literature, and demonstrates improved performance and scalability, outperforming existing methods.