On Evaluating Self-Adaptive and Self-Healing Systems using Chaos Engineering

TL;DR

This paper introduces CHESS, a chaos engineering-based approach for systematically evaluating the resilience and self-healing capabilities of self-adaptive systems, demonstrated through a smart office case study.

Contribution

It proposes a novel evaluation method using chaos engineering to assess self-healing systems, addressing the lack of systematic evaluation strategies.

Findings

CHESS effectively identifies system vulnerabilities under perturbations.

The approach reveals limitations in current self-healing mechanisms.

Lessons learned guide future improvements in self-adaptive system evaluation.

Abstract

With the growing adoption of self-adaptive systems in various domains, there is an increasing need for strategies to assess their correct behavior. In particular self-healing systems, which aim to provide resilience and fault-tolerance, often deal with unanticipated failures in critical and highly dynamic environments. Their reactive and complex behavior makes it challenging to assess if these systems execute according to the desired goals. Recently, several studies have expressed concern about the lack of systematic evaluation methods for self-healing behavior. In this paper, we propose CHESS, an approach for the systematic evaluation of self-adaptive and self-healing systems that builds on chaos engineering. Chaos engineering is a methodology for subjecting a system to unexpected conditions and scenarios. It has shown great promise in helping developers build resilient microservice architectures and cyber-physical systems. CHESS turns this idea around by using chaos engineering to evaluate how well a self-healing system can withstand such perturbations. We investigate the viability of this approach through an exploratory study on a self-healing smart office environment. The study helps us explore the promises and limitations of the approach, as well as identify directions where additional work is needed. We conclude with a summary of lessons learned.