Sparsity Exploiting Erasure Coding for Resilient Storage and Efficient I/O Access in Delta based Versioning Systems

TL;DR

This paper introduces erasure coding methods that leverage the sparsity of deltas in versioned data to enhance storage reliability and I/O efficiency in distributed systems.

Contribution

It presents novel erasure coding techniques tailored for sparse deltas in versioned storage, improving resilience and access performance.

Findings

Enhanced reliability of versioned data storage.

Improved I/O read performance through sparsity-aware coding.

Validated techniques via analytical and simulation evaluations.

Abstract

In this paper we study the problem of storing reliably an archive of versioned data. Specifically, we focus on systems where the differences (deltas) between subsequent versions rather than the whole objects are stored - a typical model for storing versioned data. For reliability, we propose erasure encoding techniques that exploit the sparsity of information in the deltas while storing them reliably in a distributed back-end storage system, resulting in improved I/O read performance to retrieve the whole versioned archive. Along with the basic techniques, we propose a few optimization heuristics, and evaluate the techniques' efficacy analytically and with numerical simulations.