Skeena: Efficient and Consistent Cross-Engine Transactions

TL;DR

Skeena introduces a comprehensive method for managing cross-engine transactions in multi-engine database systems, ensuring correctness and high performance across different data engines with up to 30x throughput improvements.

Contribution

It presents a novel lightweight snapshot tracking and commit protocol that supports various isolation levels for efficient cross-engine transactions.

Findings

Supports multiple isolation levels effectively

Achieves up to 30x throughput improvement

Maintains high performance for single-engine transactions

Abstract

Database systems are becoming increasingly multi-engine. In particular, a main-memory database engine may coexist with a traditional storage-centric engine in a system to support various applications. It is desirable to allow applications to access data in both engines using cross-engine transactions. But existing systems are either only designed for single-engine accesses, or impose many restrictions by limiting cross-engine transactions to certain isolation levels and table operations. The result is inadequate cross-engine support in terms of correctness, performance and programmability. This paper describes Skeena, a holistic approach to cross-engine transactions. We propose a lightweight snapshot tracking structure and an atomic commit protocol to efficiently ensure correctness and support various isolation levels. Evaluation results show that Skeena maintains high performance for single-engine transactions and enables cross-engine transactions which can improve throughput by up to 30x by judiciously placing tables in different engines.