Exploring high performance distributed file storage using LDPC codes

TL;DR

This paper investigates implementing a reliable, high-performance distributed storage system on commodity clusters using LDPC erasure codes, demonstrating promising results with a 50-node prototype.

Contribution

It introduces a novel distributed storage approach utilizing LDPC codes for improved reliability and efficiency on commodity hardware.

Findings

High reliability achieved with LDPC codes

Prototype system with 50 nodes shows promising performance

Distributed storage with low overhead using erasure coding

Abstract

We explore the feasibility of implementing a reliable, high performance, distributed storage system on a commodity computing cluster. Files are distributed across storage nodes using erasure coding with small Low-Density Parity-Check (LDPC) codes which provide high reliability while keeping the storage and performance overhead small. We present performance measurements done on a prototype system comprising 50 nodes which are self organised using a peer-to-peer overlay.