Energy-saving Cross-layer Optimization of Big Data Transfer Based on Historical Log Analysis

TL;DR

This paper introduces Cross-LayerHLA, a machine learning-based framework that analyzes historical logs to optimize data transfer parameters, significantly reducing energy consumption at end-systems while maintaining high throughput.

Contribution

It proposes a novel cross-layer optimization method using offline log analysis and machine learning to dynamically tune parameters for energy-efficient data transfers.

Findings

Reduces end-system energy consumption during data transfers.

Outperforms existing solutions in energy efficiency and throughput.

Utilizes offline analysis to enhance online parameter tuning.

Abstract

With the proliferation of data movement across the Internet, global data traffic per year has already exceeded the Zettabyte scale. The network infrastructure and end-systems facilitating the vast data movement consume an extensive amount of electricity, measured in terawatt-hours per year. This massive energy footprint costs the world economy billions of dollars partially due to energy consumed at the network end-systems. Although extensive research has been done on managing power consumption within the core networking infrastructure, there is little research on reducing the power consumption at the end-systems during active data transfers. This paper presents a novel cross-layer optimization framework, called Cross-LayerHLA, to minimize energy consumption at the end-systems by applying machine learning techniques to historical transfer logs and extracting the hidden relationships between different parameters affecting both the performance and resource utilization. It utilizes offline analysis to improve online learning and dynamic tuning of application-level and kernel-level parameters with minimal overhead. This approach minimizes end-system energy consumption and maximizes data transfer throughput. Our experimental results show that Cross-LayerHLA outperforms other state-of-the-art solutions in this area.