Past, Present and Future of Hadoop: A Survey

TL;DR

This survey reviews Hadoop, a framework for large-scale data storage and processing, highlighting its architecture, components, and capabilities in handling massive datasets efficiently on commodity hardware.

Contribution

It provides a comprehensive overview of Hadoop's architecture, components, and features, summarizing its evolution and current state in big data processing.

Findings

Hadoop enables scalable and fault-tolerant data processing.

HDFS efficiently manages large datasets on commodity hardware.

Hadoop's MapReduce simplifies parallel data processing.

Abstract

In this paper, a technology for massive data storage and computing named Hadoop is surveyed. Hadoop consists of heterogeneous computing devices like regular PCs abstracting away the details of parallel processing and developers can just concentrate on their computational problem. A Hadoop cluster is made of two parts: HDFs and Mapreduce. Hadoop cluster uses HDFS for data management. HDFS provides storage for input and output data in MapReduce jobs and is designed with abilities like high-fault tolerance, high-distribution capacity, and high throughput. It is also suitable for storing Terabyte data on clusters and it runs on flexible hardware like commodity devices.