Erasure Coding for Distributed Storage: An Overview

TL;DR

This paper surveys recent advances in erasure coding techniques like regenerating and locally recoverable codes, focusing on improving repair efficiency in distributed storage systems.

Contribution

It provides a comprehensive overview of the development and progress of erasure coding methods for efficient node repair over the past decade.

Findings

Introduction of regenerating codes and locally recoverable codes

Enhanced repair efficiency in distributed storage systems

Progress in repairing Reed-Solomon codes

Abstract

In a distributed storage system, code symbols are dispersed across space in nodes or storage units as opposed to time. In settings such as that of a large data center, an important consideration is the efficient repair of a failed node. Efficient repair calls for erasure codes that in the face of node failure, are efficient in terms of minimizing the amount of repair data transferred over the network, the amount of data accessed at a helper node as well as the number of helper nodes contacted. Coding theory has evolved to handle these challenges by introducing two new classes of erasure codes, namely regenerating codes and locally recoverable codes as well as by coming up with novel ways to repair the ubiquitous Reed-Solomon code. This survey provides an overview of the efforts in this direction that have taken place over the past decade.