Word Frequency Counting Based on Serverless MapReduce

TL;DR

This paper explores optimizing word frequency counting using a serverless MapReduce framework, demonstrating improved efficiency and reduced execution time through experimental analysis of function configurations.

Contribution

It introduces a serverless MapReduce approach for word counting and identifies optimal numbers of map and reduce functions to enhance performance.

Findings

Execution time decreases with more map and reduce functions.

Overall efficiency improves as function numbers increase.

Optimal function counts vary per task and workload.

Abstract

With the increasing demand for high-performance and high-efficiency computing, cloud computing, especially serverless computing, has gradually become a research hotspot in recent years, attracting numerous research attention. Meanwhile, MapReduce, which is a popular big data processing model in the industry, has been widely applied in various fields. Inspired by the serverless framework of Function as a Service and the high concurrency and robustness of MapReduce programming model, this paper focus on combining them to reduce the time span and increase the efficiency when executing the word frequency counting task. In this case, the paper use a MapReduce programming model based on a serverless computing platform to figure out the most optimized number of Map functions and Reduce functions for a particular task. For the same amount of workload, extensive experiments show both execution time reduces and the overall efficiency of the program improves at different rates as the number of map functions and reduce functions increases. This paper suppose the discovery of the most optimized number of map and reduce functions can help cooperations and programmers figure out the most optimized solutions.