Using Meta-heuristics and Machine Learning for Software Optimization of Parallel Computing Systems: A Systematic Literature Review

TL;DR

This systematic review analyzes how meta-heuristics and machine learning techniques are applied to optimize software performance in parallel computing systems, highlighting current methods, challenges, and future research directions.

Contribution

It provides a comprehensive classification and analysis of existing optimization approaches using machine learning and meta-heuristics in parallel computing.

Findings

Most approaches focus on compile-time optimization.

Machine learning methods are increasingly used for run-time tuning.

Challenges include handling system complexity and scalability.

Abstract

While modern parallel computing systems offer high performance, utilizing these powerful computing resources to the highest possible extent demands advanced knowledge of various hardware architectures and parallel programming models. Furthermore, optimized software execution on parallel computing systems demands consideration of many parameters at compile-time and run-time. Determining the optimal set of parameters in a given execution context is a complex task, and therefore to address this issue researchers have proposed different approaches that use heuristic search or machine learning. In this paper, we undertake a systematic literature review to aggregate, analyze and classify the existing software optimization methods for parallel computing systems. We review approaches that use machine learning or meta-heuristics for software optimization at compile-time and run-time. Additionally, we discuss challenges and future research directions. The results of this study may help to better understand the state-of-the-art techniques that use machine learning and meta-heuristics to deal with the complexity of software optimization for parallel computing systems. Furthermore, it may aid in understanding the limitations of existing approaches and identification of areas for improvement.