Fast Consistent Hashing in Constant Time

TL;DR

This paper introduces a novel consistent hashing algorithm that achieves constant-time key lookup and minimal key remapping, significantly improving scalability and speed for distributed systems.

Contribution

The paper presents a power consistent hash algorithm that operates in expected O(1) time without storing bucket ranges, enhancing scalability and efficiency.

Findings

Achieves expected O(1) lookup time independent of bucket count.

Uses two auxiliary hash functions for uniform distribution.

Remaps minimal keys when the number of buckets changes.

Abstract

Consistent hashing is a technique that can minimize key remapping when the number of hash buckets changes. The paper proposes a fast consistent hash algorithm (called power consistent hash) that has $O(1)$ expected time for key lookup, independent of the number of buckets. Hash values are computed in real time. No search data structure is constructed to store bucket ranges or key mappings. The algorithm has a lightweight design using $O(1)$ space with superior scalability. In particular, it uses two auxiliary hash functions to achieve distribution uniformity and $O(1)$ expected time for key lookup. Furthermore, it performs consistent hashing such that only a minimal number of keys are remapped when the number of buckets changes. Consistent hashing has a wide range of use cases, including load balancing, distributed caching, and distributed key-value stores. The proposed algorithm is faster than well-known consistent hash algorithms with $O(\log n)$ lookup time.