Evaluating Load Balancing Performance in Distributed Storage with Redundancy

TL;DR

This paper analyzes how redundancy level in distributed storage affects load balancing, showing that increasing redundancy improves load distribution exponentially up to a logarithmic scale, with implications for storage overhead and recovery.

Contribution

It provides a theoretical analysis of load balancing performance in redundant distributed storage systems, revealing how redundancy level impacts load distribution and system efficiency.

Findings

Load balance improves multiplicatively with redundancy d when d = o(log n).

Load balance improves exponentially with d when d = Θ(log n).

Redundancy reduces storage overhead multiplicatively but increases recovery load linearly.

Abstract

To facilitate load balancing, distributed systems store data redundantly. We evaluate the load balancing performance of storage schemes in which each object is stored at $d$ different nodes, and each node stores the same number of objects. In our model, the load offered for the objects is sampled uniformly at random from all the load vectors with a fixed cumulative value. We find that the load balance in a system of $n$ nodes improves multiplicatively with $d$ as long as $d = o\left(\log(n)\right)$, and improves exponentially once $d = \Theta\left(\log(n)\right)$. We show that the load balance improves in the same way with $d$ when the service choices are created with XOR's of $r$ objects rather than object replicas. In such redundancy schemes, storage overhead is reduced multiplicatively by $r$. However, recovery of an object requires downloading content from $r$ nodes. At the same time, the load balance increases additively by $r$. We express the system's load balance in terms of the maximal spacing or maximum of $d$ consecutive spacings between the ordered statistics of uniform random variables. Using this connection and the limit results on the maximal $d$-spacings, we derive our main results.