Bespoke OLAP: Synthesizing Workload-Specific One-size-fits-one Database Engines

TL;DR

Bespoke OLAP introduces an automated synthesis pipeline that creates highly optimized, workload-specific OLAP database engines, significantly outperforming general-purpose systems by leveraging iterative performance feedback and structured refinement.

Contribution

The paper presents a novel fully autonomous system for synthesizing bespoke OLAP engines tailored to specific workloads, reducing manual effort and improving performance.

Findings

Achieves order-of-magnitude speedups over DuckDB

Generates workload-specific engines within minutes to hours

Effectively integrates performance feedback for synthesis

Abstract

Modern OLAP engines are designed to support arbitrary analytical workloads, but this generality incurs structural overhead, including runtime schema interpretation, indirection layers, and abstraction boundaries, even in highly optimized systems. An engine specialized to a fixed workload can eliminate these costs and exploit workload-specific data structures and execution algorithms for substantially higher performance. Historically, constructing such bespoke engines has been economically impractical due to the high manual engineering effort. Recent advances in LLM-based code synthesis challenge this tradeoff by enabling automated system generation. However, naively prompting an LLM to produce a database engine does not yield a correct or efficient design, as effective synthesis requires systematic performance feedback, structured refinement, and careful management of deep architectural interdependencies. We present Bespoke OLAP, a fully autonomous synthesis pipeline for constructing high-performance database engines tightly tailored to a given workload. Our approach integrates iterative performance evaluation and automated validation to guide synthesis from storage to query execution. We demonstrate that Bespoke OLAP can generate a workload-specific engine from scratch within minutes to hours, achieving order-of-magnitude speedups over modern general-purpose systems such as DuckDB.