Correlation-wise Smoothing: Lightweight Knowledge Extraction for HPC Monitoring Data

TL;DR

The paper introduces Correlation-wise Smoothing, a lightweight and generic method for extracting visualizable signatures from high-dimensional HPC monitoring data, improving efficiency and interpretability over traditional techniques.

Contribution

It proposes a novel correlation-based smoothing technique that produces compact, visualizable signatures for HPC data, outperforming existing methods in size and speed.

Findings

Signatures are up to ten times smaller.

Processing is up to ten times faster.

Maintains performance comparable to state-of-the-art methods.

Abstract

Modern High-Performance Computing (HPC) and data center operators rely more and more on data analytics techniques to improve the efficiency and reliability of their operations. They employ models that ingest time-series monitoring sensor data and transform it into actionable knowledge for system tuning: a process known as Operational Data Analytics (ODA). However, monitoring data has a high dimensionality, is hardware-dependent and difficult to interpret. This, coupled with the strict requirements of ODA, makes most traditional data mining methods impractical and in turn renders this type of data cumbersome to process. Most current ODA solutions use ad-hoc processing methods that are not generic, are sensible to the sensors' features and are not fit for visualization. In this paper we propose a novel method, called Correlation-wise Smoothing (CS), to extract descriptive signatures from time-series monitoring data in a generic and lightweight way. Our CS method exploits correlations between data dimensions to form groups and produces image-like signatures that can be easily manipulated, visualized and compared. We evaluate the CS method on HPC-ODA, a collection of datasets that we release with this work, and show that it leads to the same performance as most state-of-the-art methods while producing signatures that are up to ten times smaller and up to ten times faster, while gaining visualizability, portability across systems and clear scaling properties.