An Execution Fingerprint Dictionary for HPC Application Recognition

TL;DR

This paper introduces an Execution Fingerprint Dictionary (EFD) that recognizes HPC applications quickly and accurately using minimal data, significantly reducing monitoring overhead compared to existing methods.

Contribution

The novel EFD approach identifies applications using only 2 minutes of a single system metric, achieving high accuracy with less data and complexity.

Findings

F-score above 95% with minimal data

Comparable accuracy to existing methods

Reduced data collection requirements

Abstract

Applications running on HPC systems waste time and energy if they: (a) use resources inefficiently, (b) deviate from allocation purpose (e.g. cryptocurrency mining), or (c) encounter errors and failures. It is important to know which applications are running on the system, how they use the system, and whether they have been executed before. To recognize known applications during execution on a noisy system, we draw inspiration from the way Shazam recognizes known songs playing in a crowded bar. Our contribution is an Execution Fingerprint Dictionary (EFD) that stores execution fingerprints of system metrics (keys) linked to application and input size information (values) as key-value pairs for application recognition. Related work often relies on extensive system monitoring (many system metrics collected over large time windows) and employs machine learning methods to identify applications. Our solution only uses the first 2 minutes and a single system metric to achieve F-scores above 95 percent, providing comparable results to related work but with a fraction of the necessary data and a straightforward mechanism of recognition.