Automatic Generation of a Hybrid Query Execution Engine

TL;DR

This paper explores integrating just-in-time query compilation into SQLite to enhance data processing speed, demonstrating significant performance improvements through automated code generation tools.

Contribution

It introduces automated tools for runtime code generation and successfully integrates them into SQLite, showcasing the practical benefits of hybrid query execution.

Findings

Speedups of up to 1.7x in microbenchmarks

Effective integration of JIT compilation into a popular database

Potential for improved query performance in real-world applications

Abstract

The ever-increasing need for fast data processing demands new methods for efficient query execution. Just-in-time query compilation techniques have been demonstrated to improve performance in a set of analytical tasks significantly. In this work, we investigate the possibility of adding this approach to existing database solutions and the benefits it provides. To that end, we create a set of automated tools to create a runtime code generation engine and integrate such an engine into SQLite which is one of the most popular relational databases in the world and is used in a large variety of contexts. Speedups of up to 1.7x were observed in microbenchmarks with queries involving a large number of operations.