Survey on Improved Scheduling in Hadoop MapReduce in Cloud Environments

TL;DR

This survey reviews various scheduling improvements in Hadoop MapReduce within cloud environments, highlighting potential enhancements to optimize resource allocation and job processing efficiency.

Contribution

It provides a comprehensive overview of existing scheduler improvements and offers guidelines for enhancing Hadoop scheduling in cloud settings.

Findings

Identifies limitations of default FIFO scheduler.

Discusses alternative priority-based scheduling methods.

Suggests strategies for improving scheduling efficiency.

Abstract

Cloud Computing is emerging as a new computational paradigm shift. Hadoop-MapReduce has become a powerful Computation Model for processing large data on distributed commodity hardware clusters such as Clouds. In all Hadoop implementations, the default FIFO scheduler is available where jobs are scheduled in FIFO order with support for other priority based schedulers also. In this paper we study various scheduler improvements possible with Hadoop and also provided some guidelines on how to improve the scheduling in Hadoop in Cloud Environments.