Capacity of Clustered Distributed Storage

TL;DR

This paper models the capacity of clustered distributed storage systems, analyzing how intra- and cross-cluster repair bandwidths affect reliable data storage and proposing methods to minimize cross-cluster traffic.

Contribution

It introduces a new clustered storage model that differentiates intra- and cross-cluster repair bandwidths and derives capacity expressions based on these resources.

Findings

Cross-cluster traffic can be eliminated with additional resources.

Capacity depends on storage, intra-cluster, and cross-cluster bandwidths.

Trade-offs exist between intra- and cross-cluster traffic for large storage capacities.

Abstract

A new system model reflecting the clustered structure of distributed storage is suggested to investigate bandwidth requirements for repairing failed storage nodes. Large data centers with multiple racks/disks or local networks of storage devices (e.g. sensor network) are good applications of the suggested clustered model. In realistic scenarios involving clustered storage structures, repairing storage nodes using intact nodes residing in other clusters is more bandwidth-consuming than restoring nodes based on information from intra-cluster nodes. Therefore, it is important to differentiate between intra-cluster repair bandwidth and cross-cluster repair bandwidth in modeling distributed storage. Capacity of the suggested model is obtained as a function of fundamental resources of distributed storage systems, namely, storage capacity, intra-cluster repair bandwidth and cross-cluster repair bandwidth. Based on the capacity expression, feasible sets of required resources which enable reliable storage are analyzed. It is shown that the cross-cluster traffic can be minimized to zero (i.e., intra-cluster local repair becomes possible) by allowing extra resources on storage capacity and intra-cluster repair bandwidth, according to a law specified in a closed-form. Moreover, trade-off between cross-cluster traffic and intra-cluster traffic is observed for sufficiently large storage capacity.