Fast Product-Matrix Regenerating Codes

TL;DR

This paper presents optimized Product-Matrix Regenerating codes that significantly improve encoding speeds in distributed storage, reducing network and CPU overhead during disk repair.

Contribution

The paper introduces a new optimization for Product-Matrix Regenerating codes, achieving higher encoding speeds and demonstrating their practical efficiency.

Findings

Achieved 790 MB/s encoding speed in typical settings

Optimized codes reduce network consumption during repair

Regenerating codes can operate with minimal CPU penalty

Abstract

Distributed storage systems support failures of individual devices by the use of replication or erasure correcting codes. While erasure correcting codes offer a better storage efficiency than replication for similar fault tolerance, they incur higher CPU consumption, higher network consumption and higher disk I/Os. To address these issues, codes specific to storage systems have been designed. Their main feature is the ability to repair a single lost disk efficiently. In this paper, we focus on one such class of codes that minimize network consumption during repair, namely regenerating codes. We implement the original Product-Matrix Regenerating codes as well as a new optimization we propose and show that the resulting optimized codes allow achieving 790 MB/s for encoding in typical settings. Reported speeds are significantly higher than previous studies, highlighting that regenerating codes can be used with little CPU penalty.