CoroBase: Coroutine-Oriented Main-Memory Database Engine

TL;DR

CoroBase introduces a coroutine-to-transaction paradigm in main-memory database engines, enabling inter-transaction batching and prefetching, significantly improving performance for read-intensive workloads without requiring application changes.

Contribution

It proposes a novel coroutine-to-transaction model that enhances data prefetching and batching in full database engines, overcoming limitations of prior component-focused solutions.

Findings

CoroBase achieves nearly 2x performance improvement on read-intensive workloads.

It maintains competitiveness on workloads less suited for prefetching.

The approach requires no changes to existing applications.

Abstract

Data stalls are a major overhead in main-memory database engines due to the use of pointer-rich data structures. Lightweight coroutines ease the implementation of software prefetching to hide data stalls by overlapping computation and asynchronous data prefetching. Prior solutions, however, mainly focused on (1) individual components and operations and (2) intra-transaction batching that requires interface changes, breaking backward compatibility. It was not clear how they apply to a full database engine and how much end-to-end benefit they bring under various workloads. This paper presents \corobase, a main-memory database engine that tackles these challenges with a new coroutine-to-transaction paradigm. Coroutine-to-transaction models transactions as coroutines and thus enables inter-transaction batching, avoiding application changes but retaining the benefits of prefetching. We show that on a 48-core server, CoroBase can perform close to 2x better for read-intensive workloads and remain competitive for workloads that inherently do not benefit from software prefetching.