An effective single-hop distributed hash table with high lookup performance and low traffic overhead

TL;DR

D1HT is a novel single-hop distributed hash table that achieves high lookup performance, low bandwidth overhead, and scalability, making it suitable for large-scale Internet applications and datacenters.

Contribution

This paper introduces D1HT, a new self-organizing, peer-to-peer DHT with a Quarantine mechanism, and provides comprehensive experimental and analytical comparisons to existing single-hop DHTs.

Findings

D1HT has up to ten times lower bandwidth requirements than existing DHTs.

D1HT achieves latency comparable to or better than directory servers.

D1HT is scalable to millions of users in Internet applications.

Abstract

Distributed Hash Tables (DHTs) have been used in several applications, but most DHTs have opted to solve lookups with multiple hops, to minimize bandwidth costs while sacrificing lookup latency. This paper presents D1HT, an original DHT which has a peer-to-peer and self-organizing architecture and maximizes lookup performance with reasonable maintenance traffic, and a Quarantine mechanism to reduce overheads caused by volatile peers. We implemented both D1HT and a prominent single-hop DHT, and we performed an extensive and highly representative DHT experimental comparison, followed by complementary analytical studies. In comparison with current single-hop DHTs, our results showed that D1HT consistently had the lowest bandwidth requirements, with typical reductions of up to one order of magnitude, and that D1HT could be used even in popular Internet applications with millions of users. In addition, we ran the first latency experiments comparing DHTs to directory servers, which revealed that D1HT can achieve latencies equivalent to or better than a directory server, and confirmed its greater scalability properties. Overall, our extensive set of results allowed us to conclude that D1HT can provide a very effective solution for a broad range of environments, from large-scale corporate datacenters to widely deployed Internet applications.