Clustered Network Coding for Maintenance in Practical Storage Systems

TL;DR

This paper introduces a network coding-based clustered erasure coding framework that significantly reduces bandwidth and repair time during maintenance in distributed storage systems, with practical implementation considerations.

Contribution

It presents a novel clustered network coding approach with a simple, practical implementation that halves data transfer during maintenance compared to traditional methods.

Findings

Reduces data transfer during repair by half.

Dramatically decreases repair bandwidth and time.

Demonstrated effectiveness on a public testbed.

Abstract

Classical erasure codes, e.g. Reed-Solomon codes, have been acknowledged as an efficient alternative to plain replication to reduce the storage overhead in reliable distributed storage systems. Yet, such codes experience high overhead during the maintenance process. In this paper we propose a novel erasure-coded framework especially tailored for networked storage systems. Our approach relies on the use of random codes coupled with a clustered placement strategy, enabling the maintenance of a failed machine at the granularity of multiple files. Our repair protocol leverages network coding techniques to reduce by half the amount of data transferred during maintenance, as several files can be repaired simultaneously. This approach, as formally proven and demonstrated by our evaluation on a public experimental testbed, enables to dramatically decrease the bandwidth overhead during the maintenance process, as well as the time to repair a failure. In addition, the implementation is made as simple as possible, aiming at a deployment into practical systems.