Analyzing Web Application Log Files to Find Hit Count Through the Utilization of Hadoop MapReduce in Cloud Computing Environment

TL;DR

This paper demonstrates how Hadoop MapReduce can efficiently analyze web server logs in a cloud environment to determine hit counts, leveraging parallel processing to improve response times.

Contribution

It applies Hadoop MapReduce to web log analysis for hit count extraction, showcasing improved performance through parallelization in cloud computing.

Findings

Hit counts for log file fields are successfully obtained.

Parallel processing reduces response time.

Hadoop effectively handles large web log data.

Abstract

MapReduce has been widely applied in various fields of data and compute intensive applications and also it is important programming model for cloud computing. Hadoop is an open-source implementation of MapReduce which operates on terabytes of data using commodity hardware. We have applied this Hadoop MapReduce programming model for analyzing web log files so that we could get hit count of specific web application. This system uses Hadoop file system to store log file and results are evaluated using Map and Reduce function. Experimental results show hit count for each field in log file. Also due to MapReduce runtime parallelization response time is reduced.