Risk-aware Stochastic Shortest Path

TL;DR

This paper introduces risk-aware control methods for stochastic shortest path problems on Markov decision processes, optimizing CVaR instead of expectation, with algorithms proven to be correct and feasible on moderate models.

Contribution

It presents novel algorithms based on linear programming and value iteration for risk-aware SSP, addressing the gap of risk consideration in traditional expectation-based approaches.

Findings

Algorithms are provably correct.

Risk-aware control is feasible on moderate models.

CVaR optimization improves risk management in SSP.

Abstract

We treat the problem of risk-aware control for stochastic shortest path (SSP) on Markov decision processes (MDP). Typically, expectation is considered for SSP, which however is oblivious to the incurred risk. We present an alternative view, instead optimizing conditional value-at-risk (CVaR), an established risk measure. We treat both Markov chains as well as MDP and introduce, through novel insights, two algorithms, based on linear programming and value iteration, respectively. Both algorithms offer precise and provably correct solutions. Evaluation of our prototype implementation shows that risk-aware control is feasible on several moderately sized models.