CEEMS: A Resource Manager Agnostic Energy and Emissions Monitoring Stack

TL;DR

CEEMS is an extensible, resource manager agnostic monitoring stack that provides real-time energy and emissions data for HPC and cloud workloads, supporting CPUs and GPUs, to promote energy-aware computing.

Contribution

It introduces CEEMS, a novel, extensible energy and emissions monitoring system compatible with various resource managers and hardware, integrated with open-source observability tools.

Findings

Capable of monitoring over 1400 nodes on Jean-Zay supercomputer

Supports real-time energy and emissions reporting for CPUs and GPUs

Successfully deployed with high job churn rate

Abstract

With the rapid acceleration of ML/AI research in the last couple of years, the energy consumption of the Information and Communication Technology (ICT) domain has rapidly increased. As a major part of this energy consumption is due to users' workloads, it is evident that users need to be aware of the energy footprint of their applications. Compute Energy and Emissions Monitoring Stack (CEEMS) has been designed to address this issue. CEEMS can report energy consumption and equivalent emissions of user workloads in real time for HPC and cloud platforms alike. Besides CPU energy usage, it supports reporting energy usage of workloads on NVIDIA and AMD GPU accelerators. CEEMS has been built around the prominent open-source tools in the observability eco-system like Prometheus and Grafana. CEEMS has been designed to be extensible and it allows the Data Center (DC) operators to easily define the energy estimation rules of user workloads based on the underlying hardware. This paper explains the architectural overview of CEEMS, data sources that are used to measure energy usage and estimate equivalent emissions and potential use cases of CEEMS from operator and user perspectives. Finally, the paper will conclude by describing how CEEMS deployment on the Jean-Zay supercomputing platform is capable of monitoring more than 1400 nodes that have a daily job churn rate of around 20k jobs.