Making an Embedded DBMS JIT-friendly

TL;DR

This paper demonstrates that inlining across the programming language and embedded DBMS boundary can significantly optimize performance, achieving up to 3.4x speed-up and slight improvements over SQLite.

Contribution

The paper introduces SQPyte, a meta-tracing VM that integrates RPython-implemented SQLite components with PyPy, enhancing in-process DBMS performance.

Findings

Achieved 3.4x speed-up in benchmarks

SQPyte is 2.2% faster than SQLite on TPC-H

Inlining reduces PL/DBMS boundary costs

Abstract

While database management systems (DBMSs) are highly optimized, interactions across the boundary between the programming language (PL) and the DBMS are costly, even for in-process embedded DBMSs. In this paper, we show that programs that interact with the popular embedded DBMS SQLite can be significantly optimized - by a factor of 3.4 in our benchmarks - by inlining across the PL / DBMS boundary. We achieved this speed-up by replacing parts of SQLite's C interpreter with RPython code and composing the resulting meta-tracing virtual machine (VM) - called SQPyte - with the PyPy VM. SQPyte does not compromise stand-alone SQL performance and is 2.2% faster than SQLite on the widely used TPC-H benchmark suite.