Resilient Strategies for Stochastic Systems: How Much Does It Take to Break a Winning Strategy?

TL;DR

This paper introduces the concept of resilience in stochastic systems, analyzing how much disturbance is needed to compromise strategies in Markov decision processes and stochastic games, with a focus on robustness and quantitative disturbance measures.

Contribution

It formalizes resilience in stochastic decision-making models and explores fundamental problems for Markov decision processes and stochastic games, extending to infinite disturbances and various aggregation methods.

Findings

Resilience can be quantified in stochastic systems using expectation and worst-case measures.

Fundamental problems for resilience in MDPs and stochastic games are identified and analyzed.

Methods for reasoning about infinite disturbances using frequency-based measures are proposed.

Abstract

We study the problem of resilient strategies in the presence of uncertainty. Resilient strategies enable an agent to make decisions that are robust against disturbances. In particular, we are interested in those disturbances that are able to flip a decision made by the agent. Such a disturbance may, for instance, occur when the intended action of the agent cannot be executed due to a malfunction of an actuator in the environment. In this work, we introduce the concept of resilience in the stochastic setting and present a comprehensive set of fundamental problems. Specifically, we discuss such problems for Markov decision processes with reachability and safety objectives, which also smoothly extend to stochastic games. To account for the stochastic setting, we provide various ways of aggregating the amounts of disturbances that may have occurred, for instance, in expectation or in the worst case. Moreover, to reason about infinite disturbances, we use quantitative measures, like their frequency of occurrence.