Capacity of Clustered Distributed Storage

TL;DR

This paper models the capacity of clustered distributed storage systems, analyzing how intra- and cross-cluster repair bandwidths affect storage efficiency and proposing intra-cluster repairable codes to minimize cross-cluster traffic.

Contribution

It introduces a new clustered storage model, derives its capacity as a function of key resources, and proposes intra-cluster repairable codes to reduce cross-cluster bandwidth usage.

Findings

Capacity decreases with the number of clusters as system size grows.

Feasible resource sets for reliable storage are characterized in closed form.

Zero cross-cluster traffic is achievable with additional resources, via intra-cluster repairable codes.

Abstract

A new system model reflecting the clustered structure of distributed storage is suggested to investigate interplay between storage overhead and repair bandwidth as storage node failures occur. Large data centers with multiple racks/disks or local networks of storage devices (e.g. sensor network) are good applications of the suggested clustered model. In realistic scenarios involving clustered storage structures, repairing storage nodes using intact nodes residing in other clusters is more bandwidth-consuming than restoring nodes based on information from intra-cluster nodes. Therefore, it is important to differentiate between intra-cluster repair bandwidth and cross-cluster repair bandwidth in modeling distributed storage. Capacity of the suggested model is obtained as a function of fundamental resources of distributed storage systems, namely, node storage capacity, intra-cluster repair bandwidth and cross-cluster repair bandwidth. The capacity is shown to be asymptotically equivalent to a monotonic decreasing function of number of clusters, as the number of storage nodes increases without bound. Based on the capacity expression, feasible sets of required resources which enable reliable storage are obtained in a closed-form solution. Specifically, it is shown that the cross-cluster traffic can be minimized to zero (i.e., intra-cluster local repair becomes possible) by allowing extra resources on storage capacity and intra-cluster repair bandwidth, according to the law specified in the closed-form. The network coding schemes with zero cross-cluster traffic are defined as intra-cluster repairable codes, which are shown to be a class of the previously developed locally repairable codes.