HyRES: A Hybrid Replication and Erasure Coding Approach to Data Storage

TL;DR

HyRES is a hybrid data storage scheme combining replication and erasure coding, offering improved flexibility, lower costs, and better reliability in distributed storage systems.

Contribution

HyRES introduces a novel hybrid approach that generalizes existing schemes, with theoretical analysis and simulation validation demonstrating its advantages.

Findings

Lower storage costs than pure replication

Reduced probability of file loss compared to individual schemes

Lower repair traffic than replication in large networks

Abstract

Reliability in distributed storage systems has typically focused on the design and deployment of data replication or erasure coding techniques. Although some scenarios have considered the use of replication for hot data and erasure coding for cold data in the same system, each is designed in isolation. We propose HyRES, a hybrid scheme incorporates the best characteristics of each scheme, thus, resulting in additional design flexibility and better potential performance for the system. We show that HyRES generalizes previously proposed hybrid schemes. We characterize the theoretical performance of HyRES as well as that of replication and erasure coding considering the effects of the size of the storage networks. We validate our theoretical results using simulations. These results show that HyRES can yield simultaneously lower storage costs than replication, lower probabilities of file loss than replication and erasure coding with similar worst case performance, and even lower effective repair traffic than replication when considering the network size.