Non-local redundancy: Erasure coding and dispersed replicas for robust retrieval in the Swarm peer-to-peer network

TL;DR

This paper details the implementation of erasure coding and dispersed replicas in the Swarm peer-to-peer network to enhance data robustness and retrieval reliability, including security considerations and retrieval strategies.

Contribution

It introduces a comprehensive approach to applying erasure codes in Swarm, including security parameterization, cross-neighbourhood redundancy, and retrieval methods for erasure-coded data.

Findings

Enhanced data availability through erasure coding

Security levels tailored for data robustness

Effective retrieval strategies for erasure-coded files

Abstract

This paper describes in detail how erasure codes are implemented in the Swarm system. First, in Section 1, we introduce erasure codes, and show how to apply them to files in Swarm (Section 2). In Section 3, we introduce security levels of data availability and derive their respective parameterisations. In Section 4, we describe a construct that enables cross-neighbourhood redundancy for singleton chunks and which completes erasure coding. Finally, in 5, we propose a number of retrieval strategies applicable to erasure-coded files.