Novel Dynamic Load Balancing Algorithm for Cloud-Based Big Data Analytics

TL;DR

This paper introduces a new dynamic load balancing algorithm for cloud-based big data analytics that reduces response time and complexity through mathematical optimization and hill-climbing techniques.

Contribution

It presents a novel load balancing algorithm combining optimization models and hill-climbing to improve efficiency in cloud big data processing.

Findings

Significant reduction in response time compared to existing algorithms

Improved throughput and request distribution metrics

Reduced complexity in load balancing process

Abstract

Big data analytics in cloud environments introduces challenges such as real-time load balancing besides security, privacy, and energy efficiency. In this paper, we propose a novel load balancing algorithm in cloud environments that performs resource allocation and task scheduling efficiently. The proposed load balancer reduces the execution response time in big data applications performed on clouds. Scheduling, in general, is an NP-hard problem. In our proposed algorithm, we provide solutions to reduce the search area that leads to reduced complexity of the load balancing. We recommend two mathematical optimization models to perform dynamic resource allocation to virtual machines and task scheduling. The provided solution is based on the hill-climbing algorithm to minimize response time. We evaluate the performance of proposed algorithms in terms of response time, turnaround time, throughput metrics, and request distribution with some of the existing algorithms that show significant improvements