MPCDF HPC Performance Monitoring System: Enabling Insight via Job-Specific Analysis
Luka Stanisic, Klaus Reuter

TL;DR
This paper presents a comprehensive HPC performance monitoring system at MPCDF that collects, aggregates, and visualizes job-specific metrics, aiding stakeholders in performance analysis and system management.
Contribution
It introduces a lightweight, open-source middleware and a scalable architecture for detailed, real-time HPC performance monitoring and reporting.
Findings
Enables detailed per-job performance analysis
Improves system management and user support
Demonstrates scalability on large HPC systems
Abstract
This paper reports on the design and implementation of the HPC performance monitoring system deployed to continuously monitor performance metrics of all jobs on the HPC systems at the Max Planck Computing and Data Facility (MPCDF). Thereby it reveals important information to various stakeholders, in particular to users, application support, system administrators, and management. On each compute node, hardware and software performance monitoring data is collected by our newly developed lightweight open-source hpcmd middleware which builds upon standard Linux tools. The data is transported via rsyslog, and aggregated and processed by a Splunk system, enabling detailed per-cluster and per-job interactive analysis in a web browser. Additionally, performance reports are provided to the users as PDF files. Finally, we report on practical experience and benefits from large-scale deployments on…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
