Design of an energy aware petaflops class high performance cluster based on power architecture

TL;DR

This paper introduces D.A.V.I.D.E., an energy-efficient petaflops-class HPC cluster with innovative hardware and software features for power monitoring, management, and application profiling, aimed at high performance and energy savings.

Contribution

It presents a novel HPC cluster design integrating advanced hardware components, custom power monitoring hardware, and software for fine-grain power management and energy-aware computing.

Findings

Design of a scalable power monitoring interface

Implementation of energy-aware job scheduling

Prototype deployment demonstrating HPC application performance

Abstract

In this paper we present D.A.V.I.D.E. (Development for an Added Value Infrastructure Designed in Europe), an innovative and energy efficient High Performance Computing cluster designed by E4 Computer Engineering for PRACE (Partnership for Advanced Computing in Europe). D.A.V.I.D.E. is built using best-in-class components (IBM's POWER8-NVLink CPUs, NVIDIA TESLA P100 GPUs, Mellanox InfiniBand EDR 100 Gb/s networking) plus custom hardware and an innovative system middleware software. D.A.V.I.D.E. features (i) a dedicated power monitor interface, built around the BeagleBone Black Board that allows high frequency sampling directly from the power backplane and scalable integration with the internal node telemetry and system level power management software; (ii) a custom-built chassis, based on OpenRack form factor, and liquid cooling that allows the system to be used in modern, energy efficient, datacenter; (iii) software components designed for enabling fine grain power monitoring, power management (i.e. power capping and energy aware job scheduling) and application power profiling, based on dedicated machine learning components. Software APIs are offered to developers and users to tune the computing node performance and power consumption around on the application requirements. The first pilot system that we will deploy at the beginning of 2017, will demonstrate key HPC applications from different fields ported and optimized for this innovative platform.