On the Power Saving in High-Speed Ethernet-based Networks for Supercomputers and Data Centers

TL;DR

This paper analyzes power-saving techniques in high-speed Ethernet networks for supercomputers and data centers, proposing improvements to the PerfBound mechanism to reduce energy use with minimal performance impact.

Contribution

It provides the first thorough analysis of PerfBound, proposes enhancements, and evaluates power-saving effects on emerging high-performance interconnects.

Findings

PerfBound can significantly reduce energy consumption.

Application traffic patterns influence network energy efficiency.

Proposed improvements to PerfBound minimize performance degradation.

Abstract

The increase in computation and storage has led to a significant growth in the scale of systems powering applications and services, raising concerns about sustainability and operational costs. In this paper, we explore power-saving techniques in high-performance computing (HPC) and datacenter networks, and their relation with performance degradation. From this premise, we propose leveraging Energy Efficient Ethernet (EEE) protocol, with the flexibility to extend to conventional Ethernet or upcoming Ethernet-derived interconnect versions of BXI and Omnipath. We analyze the PerfBound power-saving mechanism, identifying possible improvements and modeling it into a simulation framework. Through different experiments, we examine its impact on performance and determine the most appropriate interconnect. We also study traffic patterns generated by selected HPC and machine learning applications to evaluate the behavior of power-saving techniques. From these experiments, we provide an analysis of how applications affect system and network energy consumption. Based on this, we disclose the weakness of dynamic power-down mechanisms and propose an approach that improves energy reduction with minimal or no performance penalty. To the best of our knowledge, this work presents the first thorough analysis of PerfBound and an enhancement to the technique, while also targeting emerging post-exascale networks.