Decentralized Erasure Codes for Distributed Networked Storage

TL;DR

This paper introduces Decentralized Erasure Codes, a novel class of linear codes inspired by network coding, designed for efficient distributed storage with minimal communication and computation costs.

Contribution

It presents a new decentralized coding scheme that is optimally sparse and suitable for distributed networked storage, improving efficiency over traditional random linear coding.

Findings

Decentralized Erasure Codes are optimally sparse.

They reduce communication, storage, and computation costs.

The codes enable data retrieval by querying any k storage nodes.

Abstract

We consider the problem of constructing an erasure code for storage over a network when the data sources are distributed. Specifically, we assume that there are n storage nodes with limited memory and k<n sources generating the data. We want a data collector, who can appear anywhere in the network, to query any k storage nodes and be able to retrieve the data. We introduce Decentralized Erasure Codes, which are linear codes with a specific randomized structure inspired by network coding on random bipartite graphs. We show that decentralized erasure codes are optimally sparse, and lead to reduced communication, storage and computation cost over random linear coding.