Conditional Value-at-Risk for Reachability and Mean Payoff in Markov Decision Processes

TL;DR

This paper explores the application of Conditional Value-at-Risk (CVaR) in Markov decision processes to quantify and manage risk in reachability and mean-payoff objectives, providing complexity bounds and strategy characterizations.

Contribution

It introduces CVaR constraints into MDPs, analyzes their computational complexity, and characterizes the structure of optimal strategies with respect to memory and randomization.

Findings

Derived bounds on decision problem complexity.

Characterized strategy structures for CVaR constraints.

Extended analysis to conjunctions with expectation and VaR constraints.

Abstract

We present the conditional value-at-risk (CVaR) in the context of Markov chains and Markov decision processes with reachability and mean-payoff objectives. CVaR quantifies risk by means of the expectation of the worst p-quantile. As such it can be used to design risk-averse systems. We consider not only CVaR constraints, but also introduce their conjunction with expectation constraints and quantile constraints (value-at-risk, VaR). We derive lower and upper bounds on the computational complexity of the respective decision problems and characterize the structure of the strategies in terms of memory and randomization.