In Search of a Key Value Store with High Performance and High Availability

TL;DR

This paper introduces LogStore, a high-performance, high-availability write-once key-value store optimized for write-heavy workloads, using a log-as-database architecture and a single-threaded partition model.

Contribution

The paper presents LogStore, a novel key-value system that achieves high performance and availability without relying on specialized hardware or external systems.

Findings

LogStore demonstrates excellent write performance in empirical tests.

It achieves fast recovery and replication processes.

The system effectively minimizes IO costs through optimized buffer management.

Abstract

In recent year, the write-heavy applications is more and more prevalent. How to efficiently handle this sort of workload is one of intensive research direction in the field of database system. The overhead caused by write operation is mainly issued by two reasons: 1) the hardware level, i.e., the IO cost caused by logging. We can't remove this cost in short period 2) the dual-copy software architecture and serial replay. The born of log as database architecture is originated to overcome the software defect. But existing systems treating log as database either are built on top of special infrastructure such as infiniband or NVRam (Non-Volatile Random access memory) which is far from widely available or are constructed with the help of other system such as Dynamo which is lack of flexibility. In this paper we build only write-once key-value system called LogStore from scratch to support our instant messenger business. The key features of LogStore include: 1) a single thread per partition executing mode, which eliminates the concurrency overhead; 2) log as database to enable write-once feature and freshness on the standby. We achieve high availability by embedding replication protocol other than dependent on other infrastructure; 3) fine-grained and low overhead data buffer pool management to effectively minimize IO cost. According to our empirical evaluations LogStore has good performance in write operation, recovery and replication