Management of Data Replication for PC Cluster-based Cloud Storage System

TL;DR

This paper presents an enhanced data replication management scheme for a cost-effective PC cluster-based cloud storage system using HDFS, optimizing data distribution, balancing, and fault tolerance.

Contribution

It introduces a novel replication management scheme that improves data balancing and fault tolerance in inexpensive PC cluster cloud storage systems.

Findings

Storage balancing depends on disk space and node failure probability.

Optimized replica number improves data availability.

System effectively balances load across commodity nodes.

Abstract

Storage systems are essential building blocks for cloud computing infrastructures. Although high performance storage servers are the ultimate solution for cloud storage, the implementation of inexpensive storage system remains an open issue. To address this problem, the efficient cloud storage system is implemented with inexpensive and commodity computer nodes that are organized into PC cluster based datacenter. Hadoop Distributed File System (HDFS) is an open source cloud based storage platform and designed to be deployed in low-cost hardware. PC Cluster based Cloud Storage System is implemented with HDFS by enhancing replication management scheme. Data objects are distributed and replicated in a cluster of commodity nodes located in the cloud. This system provides optimum replica number as well as weighting and balancing among the storage server nodes. The experimental results show that storage can be balanced depending on the available disk space, expected availability and failure probability of each node in PC cluster.