MetaHive: A Cache-Optimized Metadata Management for Heterogeneous Key-Value Stores

TL;DR

MetaHive introduces a cache-optimized metadata management system for heterogeneous cloud key-value stores, improving data integrity and retrieval efficiency without significant performance overhead.

Contribution

It presents a novel disaggregated metadata management approach that enhances cache utilization and data integrity in heterogeneous KV store clusters.

Findings

MetaHive improves metadata retrieval efficiency.

It maintains data integrity with minimal performance impact.

Demonstrated effectiveness in RocksDB deployments.

Abstract

Cloud key-value (KV) stores provide businesses with a cost-effective and adaptive alternative to traditional on-premise data management solutions. KV stores frequently consist of heterogeneous clusters, characterized by varying hardware specifications of the deployment nodes, with each node potentially running a distinct version of the KV store software. This heterogeneity is accompanied by the diverse metadata that they need to manage. In this study, we introduce MetaHive, a cache-optimized approach to managing metadata in heterogeneous KV store clusters. MetaHive disaggregates the original data from its associated metadata to promote independence between them, while maintaining their interconnection during usage. This makes the metadata opaque from the downstream processes and the other KV stores in the cluster. MetaHive also ensures that the KV and metadata entries are stored in the vicinity of each other in memory and storage. This allows MetaHive to optimally utilize the caching mechanism without extra storage read overhead for metadata retrieval. We deploy MetaHive to ensure data integrity in RocksDB and demonstrate its rapid data validation with minimal effect on performance.