Incremental View Maintenance with Triple Lock Factorization Benefits

TL;DR

This paper presents F-IVM, a higher-order incremental view maintenance framework that efficiently handles various data tasks by factorizing computations, outperforming existing methods significantly in speed and memory usage.

Contribution

F-IVM introduces a unified higher-order IVM approach with triple lock factorization, enabling efficient incremental maintenance across multiple tasks.

Findings

F-IVM outperforms classical IVM and recomputation methods by orders of magnitude.

F-IVM uses less memory while maintaining high efficiency.

F-IVM is applicable to diverse tasks like gradient computation and query evaluation.

Abstract

We introduce F-IVM, a unified incremental view maintenance (IVM) approach for a variety of tasks, including gradient computation for learning linear regression models over joins, matrix chain multiplication, and factorized evaluation of conjunctive queries. F-IVM is a higher-order IVM algorithm that reduces the maintenance of the given task to the maintenance of a hierarchy of increasingly simpler views. The views are functions mapping keys, which are tuples of input data values, to payloads, which are elements from a task-specific ring. Whereas the computation over the keys is the same for all tasks, the computation over the payloads depends on the task. F-IVM achieves efficiency by factorizing the computation of the keys, payloads, and updates. We implemented F-IVM as an extension of DBToaster. We show in a range of scenarios that it can outperform classical first-order IVM, DBToaster's fully recursive higher-order IVM, and plain recomputation by orders of magnitude while using less memory.