Network Traffic Driven Storage Repair

TL;DR

This paper presents explicit constructions of locally repairable and regenerating codes based on HashTag codes, emphasizing network traffic-driven repair strategies that optimize bandwidth, I/O, and access time, with practical implementation insights.

Contribution

It introduces explicit HashTag code-based constructions with dual repair strategies, enhancing repair efficiency and system performance in distributed storage.

Findings

Dual repair strategies improve efficiency in practical systems

Algorithms for efficient global parity node repair are provided

Implementation in Hadoop demonstrates practical benefits

Abstract

Recently we constructed an explicit family of locally repairable and locally regenerating codes. Their existence was proven by Kamath et al. but no explicit construction was given. Our design is based on HashTag codes that can have different sub-packetization levels. In this work we emphasize the importance of having two ways to repair a node: repair only with local parity nodes or repair with both local and global parity nodes. We say that the repair strategy is network traffic driven since it is in connection with the concrete system and code parameters: the repair bandwidth of the code, the number of I/O operations, the access time for the contacted parts and the size of the stored file. We show the benefits of having repair duality in one practical example implemented in Hadoop. We also give algorithms for efficient repair of the global parity nodes.