Allocations for Heterogenous Distributed Storage

TL;DR

This paper addresses the challenge of optimally allocating data across heterogeneous storage nodes with varying reliabilities to maximize recovery probability, using approximation algorithms and convex optimization.

Contribution

It introduces an approximation framework for the complex allocation problem in heterogeneous storage systems, leveraging large deviation inequalities and convex optimization techniques.

Findings

Proposes two approximation algorithms for storage allocation.

Analyzes asymptotic performance of the proposed allocations.

Provides a scalable approach for heterogeneous storage reliability optimization.

Abstract

We study the problem of storing a data object in a set of data nodes that fail independently with given probabilities. Our problem is a natural generalization of a homogenous storage allocation problem where all the nodes had the same reliability and is naturally motivated for peer-to-peer and cloud storage systems with different types of nodes. Assuming optimal erasure coding (MDS), the goal is to find a storage allocation (i.e, how much to store in each node) to maximize the probability of successful recovery. This problem turns out to be a challenging combinatorial optimization problem. In this work we introduce an approximation framework based on large deviation inequalities and convex optimization. We propose two approximation algorithms and study the asymptotic performance of the resulting allocations.