Offset-value coding in database query processing
Goetz Graefe, Thanh Do

TL;DR
This paper introduces a new method for efficiently generating offset-value codes in database query processing, enabling faster execution of sorting, grouping, and join operations by avoiding row-by-row comparisons.
Contribution
It presents a novel theorem and algorithms that produce offset-value codes without pairwise row comparisons, improving efficiency in order-preserving query operators.
Findings
Significantly speeds up query execution in systems like Google's Napa and F1.
Enables order-preserving algorithms to compute offset-value codes efficiently.
Reduces computational overhead in producing offset-value codes for query pipelines.
Abstract
Recent work shows how offset-value coding speeds up database query execution, not only sorting but also duplicate removal and grouping (aggregation) in sorted streams, order-preserving exchange (shuffle), merge join, and more. It already saves thousands of CPUs in Google's Napa and F1 Query systems, e.g., in grouping algorithms and in log-structured merge-forests. In order to realize the full benefit of interesting orderings, however, query execution algorithms must not only consume and exploit offset-value codes but also produce offset-value codes for the next operator in the pipeline. Our research has sought ways to produce offset-value codes without comparing successive output rows one-by-one, column-by-column. This short paper introduces a new theorem and, based on its proof and a simple corollary, describes in detail how order-preserving algorithms (from filter to merge join and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Database Systems and Queries · Algorithms and Data Compression · Advanced Data Storage Technologies
