Energy-efficient Analytics for Geographically Distributed Big Data

TL;DR

This paper proposes a dynamic global manager selection algorithm to optimize energy efficiency in geo-distributed big data analytics, balancing energy cost and latency through real-time stochastic optimization.

Contribution

It introduces a novel GMSA algorithm that adaptively minimizes energy consumption by leveraging system diversity and real-time data in geo-distributed environments.

Findings

GMSA reduces energy costs significantly in simulations.

The algorithm balances energy efficiency and latency effectively.

Simulation results confirm the approach's practicality.

Abstract

Big data analytics on geographically distributed datasets (across data centers or clusters) has been attracting increasing interests from both academia and industry, but also significantly complicates the system and algorithm designs. In this article, we systematically investigate the geo-distributed big-data analytics framework by analyzing the fine-grained paradigm and the key design principles. We present a dynamic global manager selection algorithm (GMSA) to minimize energy consumption cost by fully exploiting the system diversities in geography and variation over time. The algorithm makes real-time decisions based on the measurable system parameters through stochastic optimization methods, while achieving the performance balances between energy cost and latency. Extensive trace-driven simulations verify the effectiveness and efficiency of the proposed algorithm. We also highlight several potential research directions that remain open and require future elaborations in analyzing geo-distributed big data.