Applying Machine Learning in Self-Adaptive Systems: A Systematic Literature Review

TL;DR

This systematic review explores how machine learning is integrated into self-adaptive systems, focusing on techniques, challenges, and future directions within the MAPE framework to guide researchers and practitioners.

Contribution

It provides a comprehensive overview of the state of the art in applying machine learning to self-adaptive systems, highlighting key methods, challenges, and proposing an initial design process.

Findings

Machine learning is mainly used for updating adaptation rules and resource management.

Supervised and reinforcement learning are the most common methods.

Unsupervised learning is underutilized despite its suitability for automation.

Abstract

Recently, we witness a rapid increase in the use of machine learning in self-adaptive systems. Machine learning has been used for a variety of reasons, ranging from learning a model of the environment of a system during operation to filtering large sets of possible configurations before analysing them. While a body of work on the use of machine learning in self-adaptive systems exists, there is currently no systematic overview of this area. Such overview is important for researchers to understand the state of the art and direct future research efforts. This paper reports the results of a systematic literature review that aims at providing such an overview. We focus on self-adaptive systems that are based on a traditional Monitor-Analyze-Plan-Execute feedback loop (MAPE). The research questions are centred on the problems that motivate the use of machine learning in self-adaptive systems, the key engineering aspects of learning in self-adaptation, and open challenges. The search resulted in 6709 papers, of which 109 were retained for data collection. Analysis of the collected data shows that machine learning is mostly used for updating adaptation rules and policies to improve system qualities, and managing resources to better balance qualities and resources. These problems are primarily solved using supervised and interactive learning with classification, regression and reinforcement learning as the dominant methods. Surprisingly, unsupervised learning that naturally fits automation is only applied in a small number of studies. Key open challenges in this area include the performance of learning, managing the effects of learning, and dealing with more complex types of goals. From the insights derived from this systematic literature review we outline an initial design process for applying machine learning in self-adaptive systems that are based on MAPE feedback loops.