Blitzcrank: Fast Semantic Compression for In-memory Online Transaction Processing

TL;DR

BLITZCRANK is a novel high-speed semantic compression technique for in-memory OLTP databases that significantly reduces memory usage while maintaining high throughput and low latency.

Contribution

It introduces new semantic models and a delayed coding entropy algorithm, enabling fast compression and decompression for dynamic, row-store data.

Findings

Achieves sub-microsecond decompression latency.

Provides 85% memory reduction in TPC-C benchmark.

Maintains high throughput with moderate performance impact.

Abstract

We present BLITZCRANK, a high-speed semantic compressor designed for OLTP databases. Previous solutions are inadequate for compressing row-stores: they suffer from either low compression factor due to a coarse compression granularity or suboptimal performance due to the inefficiency in handling dynamic data sets. To solve these problems, we first propose novel semantic models that support fast inferences and dynamic value set for both discrete and continuous data types. We then introduce a new entropy encoding algorithm, called delayed coding, that achieves significant improvement in the decoding speed compared to modern arithmetic coding implementations. We evaluate BLITZCRANK in both standalone microbenchmarks and a multicore in-memory row-store using the TPC-C benchmark. Our results show that BLITZCRANK achieves a sub-microsecond latency for decompressing a random tuple while obtaining high compression factors. This leads to an 85% memory reduction in the TPC-C evaluation with a moderate (19%) throughput degradation. For data sets larger than the available physical memory, BLITZCRANK help the database sustain a high throughput for more transactions before the l/O overhead dominates.