HiStore: Rethinking Hybrid Index in RDMA-based Key-Value Store

TL;DR

HiStore introduces a hybrid index combining hash tables and skiplists for RDMA-based key-value stores, significantly improving range query efficiency and overall performance.

Contribution

It proposes a novel hybrid index design for RDMA key-value stores, integrating hash tables and skiplists with asynchronous updates for better range query support.

Findings

GET and SCAN operations improved by up to 2.03x

Efficient support for range queries with hybrid index

Enhanced performance and high availability in RDMA key-value stores

Abstract

RDMA (Remote Direct Memory Access) is widely exploited in building key-value stores to achieve ultra low latency. In RDMA-based key-value stores, the indexing time takes a large fraction (up to 74%) of the overall operation latency as RDMA enables fast data accesses. However, the single index structure used in existing RDMA-based key-value stores, either hash-based or sorted index, fails to support range queries efficiently while achieving high performance for single-point operations. In this paper, we reconsider the adoption of hybrid index in the key-value stores based on RDMA, to combine the benefits of hash table and sorted index. We propose HiStore, an RDMA-based key-value store using hash table for single-point lookups and leveraging skiplist for range queries. To maintain strong consistency in a lightweight and efficient approach, HiStore introduces index groups where a skiplist corresponds to a hash table, and asynchronously applies updates to the skiplist within a group. Guided by previous work on using RDMA for key-value services, HiStore dedicatedly chooses different RDMA primitives to optimize the read and write performance. Furthermore, HiStore tolerates the failures of servers that maintain index structures with index replication for high availability. Our evaluation results demonstrate that HiStore improves the performance of both GET and SCAN operations (by up to 2.03x) with hybrid index.