Formal Quality Measures for Predictors in Markov Decision Processes

TL;DR

This paper introduces quantitative measures to evaluate the effectiveness of predictors in Markov decision processes, focusing on their ability to predict failures across all memoryless policies, enhancing system reliability.

Contribution

It proposes novel quantitative notions for predictor quality in MDPs, inspired by statistical analysis and causality concepts, addressing a gap in predictive system assessment.

Findings

New measures for predictor effectiveness in MDPs.

Evaluation of predictor quality across all memoryless policies.

Application of statistical and causality-inspired metrics.

Abstract

In adaptive systems, predictors are used to anticipate changes in the systems state or behavior that may require system adaption, e.g., changing its configuration or adjusting resource allocation. Therefore, the quality of predictors is crucial for the overall reliability and performance of the system under control. This paper studies predictors in systems exhibiting probabilistic and non-deterministic behavior modelled as Markov decision processes (MDPs). Main contributions are the introduction of quantitative notions that measure the effectiveness of predictors in terms of their average capability to predict the occurrence of failures or other undesired system behaviors. The average is taken over all memoryless policies. We study two classes of such notions. One class is inspired by concepts that have been introduced in statistical analysis to explain the impact of features on the decisions of binary classifiers (such as precision, recall, f-score). Second, we study a measure that borrows ideas from recent work on probability-raising causality in MDPs and determines the quality of a predictor by the fraction of memoryless policies under which (the set of states in) the predictor is a probability-raising cause for the considered failure scenario.