Sharding Distributed Databases: A Critical Review

TL;DR

This paper critically reviews the challenges of sharding in distributed databases, focusing on consensus issues affecting scalability and performance, and evaluates current solutions including classical systems and DLTs.

Contribution

It provides a comprehensive analysis of sharding challenges and reviews existing implementations in distributed databases and DLTs to address scalability and performance.

Findings

Consensus mechanisms cause message complexity limiting scalability

Sharding can mitigate some scalability issues in distributed systems

Review of classical and DLT sharding approaches

Abstract

This article examines the significant challenges encountered in implementing sharding within distributed replication systems. It identifies the impediments of achieving consensus among large participant sets, leading to scalability, throughput, and performance limitations. These issues primarily arise due to the message complexity inherent in consensus mechanisms. In response, we investigate the potential of sharding to mitigate these challenges, analyzing current implementations within distributed replication systems. Additionally, we offer a comprehensive review of replication systems, encompassing both classical distributed databases as well as Distributed Ledger Technologies (DLTs) employing sharding techniques. Through this analysis, the article aims to provide insights into addressing the scalability and performance concerns in distributed replication systems.