Tempura: A General Cost Based Optimizer Framework for Incremental Data Processing (Extended Version)
Zuozhi Wang, Kai Zeng, Botong Huang, Wei Chen, Xiaozong Cui, Bo Wang,, Ji Liu, Liya Fan, Dachuan Qu, Zhenyu Hou, Tao Guan, Chen Li, Jingren Zhou

TL;DR
Tempura is a comprehensive cost-based optimizer framework for incremental data processing that unifies existing techniques and enables optimal plan generation across diverse scenarios.
Contribution
It introduces the TIP model for formal incremental query planning and provides a flexible framework for optimizing incremental data processing.
Findings
Tempura effectively generates optimal incremental plans in various scenarios.
Experimental results demonstrate Tempura's efficiency and effectiveness.
The framework unifies multiple existing techniques for incremental processing.
Abstract
Incremental processing is widely-adopted in many applications, ranging from incremental view maintenance, stream computing, to recently emerging progressive data warehouse and intermittent query processing. Despite many algorithms developed on this topic, none of them can produce an incremental plan that always achieves the best performance, since the optimal plan is data dependent. In this paper, we develop a novel cost-based optimizer framework, called Tempura, for optimizing incremental data processing. We propose an incremental query planning model called TIP based on the concept of time-varying relations, which can formally model incremental processing in its most general form. We give a full specification of Tempura, which can not only unify various existing techniques to generate an optimal incremental plan, but also allow the developer to add their rewrite rules. We study how to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Database Systems and Queries · Data Management and Algorithms · Semantic Web and Ontologies
