Learning Recovery Strategies for Dynamic Self-healing in Reactive Systems

TL;DR

This paper introduces a novel self-healing framework for reactive systems that uses expressive runtime monitors and reinforcement learning to dynamically detect failures and execute recovery strategies, improving resilience.

Contribution

It proposes a new approach combining expressive runtime monitors with reinforcement learning to learn and execute recovery strategies in complex reactive systems.

Findings

Achieved 55%-92% failure recovery rate in the mouse movement tracking application.

Performed on par with predefined strategies in the DeltaIoT self-healing system.

Demonstrated feasibility of dynamic recovery strategy extraction and execution.

Abstract

Self-healing systems depend on following a set of predefined instructions to recover from a known failure state. Failure states are generally detected based on domain specific specialized metrics. Failure fixes are applied at predefined application hooks that are not sufficiently expressive to manage different failure types. Self-healing is usually applied in the context of distributed systems, where the detection of failures is constrained to communication problems, and resolution strategies often consist of replacing complete components. Our proposal targets complex reactive systems, defining monitors as predicates specifying satisfiability conditions of system properties. Such monitors are functionally expressive and can be defined at run time to detect failure states at any execution point. Once failure states are detected, we use a Reinforcement Learning-based technique to learn a recovery strategy based on users' corrective sequences. Finally, to execute the learned strategies, we extract them as COP variations that activate dynamically whenever the failure state is detected, overwriting the base system behavior with the recovery strategy for that state. We validate the feasibility and effectiveness of our framework through a prototypical reactive application for tracking mouse movements, and the DeltaIoT exemplar for self-healing systems. Our results demonstrate that with just the definition of monitors, the system is effective in detecting and recovering from failures between 55%-92% of the cases in the first application, and at par with the predefined strategies in the second application.