Incremental View Maintenance For Collection Programming

TL;DR

This paper introduces an efficient incremental view maintenance technique for positive nested relational calculus on bags, enabling faster updates in large dynamic datasets by translating general queries into efficiently incrementalizable fragments.

Contribution

It presents the first solution for incrementalizing NRC+ on bags, models operator costs, classifies queries, and provides a translation to efficiently incrementalizable queries.

Findings

IncNRC+ is efficiently incrementalizable

Incremental maintenance for NRC+ is in NC0 complexity class

Recursive IVM can be applied to IncNRC+ for speedups

Abstract

In the context of incremental view maintenance (IVM), delta query derivation is an essential technique for speeding up the processing of large, dynamic datasets. The goal is to generate delta queries that, given a small change in the input, can update the materialized view more efficiently than via recomputation. In this work we propose the first solution for the efficient incrementalization of positive nested relational calculus (NRC+) on bags (with integer multiplicities). More precisely, we model the cost of NRC+ operators and classify queries as efficiently incrementalizable if their delta has a strictly lower cost than full re-evaluation. Then, we identify IncNRC+; a large fragment of NRC+ that is efficiently incrementalizable and we provide a semantics-preserving translation that takes any NRC+ query to a collection of IncNRC+ queries. Furthermore, we prove that incremental maintenance for NRC+ is within the complexity class NC0 and we showcase how recursive IVM, a technique that has provided significant speedups over traditional IVM in the case of flat queries [25], can also be applied to IncNRC+.