A Review of Tools and Techniques for Optimization of Workload Mapping and Scheduling in Heterogeneous HPC System

TL;DR

This paper systematically reviews workload mapping and scheduling tools in heterogeneous HPC systems, analyzing recent research, identifying limitations of current methods, and proposing hybrid optimization strategies for improved performance.

Contribution

It provides a comprehensive analysis of recent tools and techniques, highlighting gaps and advocating for hybrid approaches combining heuristics, machine learning, and quantum computing.

Findings

Conventional scheduling models often lack expressiveness for modern HPC complexity.

HPC scheduling problems are NP-hard due to their combinatorial nature.

Hybrid optimization approaches can enhance scalability and efficiency.

Abstract

This paper presents a systematic review of mapping and scheduling strategies within the High-Performance Computing (HPC) compute continuum, with a particular emphasis on heterogeneous systems. It introduces a prototype workflow to establish foundational concepts in workload characterization and resource allocation. Building on this, a thorough analysis of 66 selected research papers - spanning the period from 2017 to 2024 - is conducted, evaluating contemporary tools and techniques used for workload mapping and scheduling. The review highlights that conventional Job Shop scheduling formulations often lack the expressiveness required to model the complexity of modern HPC data centers effectively. It also reaffirms the classification of HPC scheduling problems as NP-hard, due to their combinatorial nature and the diversity of system and workload constraints. The analysis reveals a prevailing reliance on heuristic and meta-heuristic strategies, including nature-inspired, evolutionary, sorting, and search algorithms. To bridge the observed gaps, the study advocates for hybrid optimization approaches that strategically integrate heuristics, meta-heuristics, machine learning, and emerging quantum computing techniques. Such integration, when tailored to specific problem domains, holds promise for significantly improving the scalability, efficiency, and adaptability of workload optimization in heterogeneous HPC environments.