Network Traffic Analysis:Hadoop Pig vs Typical MapReduce

TL;DR

This paper compares Hadoop Pig and traditional MapReduce for network traffic analysis, highlighting Hadoop's efficiency in processing large-scale Netflow and web log data in big data environments.

Contribution

It provides a comparative analysis of Hadoop Pig and MapReduce, demonstrating Hadoop's advantages in handling big data network traffic analysis tasks.

Findings

Hadoop Pig offers simpler programming compared to MapReduce.

Hadoop efficiently processes large network traffic datasets.

Pig improves development speed for big data analysis.

Abstract

Big data analysis has become much popular in the present day scenario and the manipulation of big data has gained the keen attention of researchers in the field of data analytics. Analysis of big data is currently considered as an integral part of many computational and statistical departments. As a result, novel approaches in data analysis are evolving on a daily basis. Thousands of transaction requests are handled and processed everyday by different websites associated with e-commerce, e-banking, e-shopping carts etc. The network traffic and weblog analysis comes to play a crucial role in such situations where Hadoop can be suggested as an efficient solution for processing the Netflow data collected from switches as well as website access-logs during fixed intervals.