On Optimizing the Conditional Value-at-Risk of a Maximum Cost for Risk-Averse Safety Analysis

TL;DR

This paper introduces a novel approach to safety analysis by optimizing the Conditional Value-at-Risk of maximum costs in stochastic control systems, providing a more comprehensive risk measure than traditional methods.

Contribution

It formulates and solves a new CVaR-based control problem for maximum costs, with a dynamic programming solution and proof of optimal policy existence.

Findings

Proposed a dynamic programming framework for CVaR optimization.

Proved the equivalence of CVaR with dynamic program solutions.

Demonstrated the approach on sewer overflow risk assessment.

Abstract

The popularity of Conditional Value-at-Risk (CVaR), a risk functional from finance, has been growing in the control systems community due to its intuitive interpretation and axiomatic foundation. We consider a nonstandard optimal control problem in which the goal is to minimize the CVaR of a maximum random cost subject to a Borel-space Markov decision process. The objective represents the maximum departure from a desired operating region averaged over a given fraction of the worst cases. This problem provides a safety criterion for a stochastic system that is informed by both the probability and severity of the potential consequences of the system's behavior. In contrast, existing safety analysis frameworks apply stage-wise risk constraints or assess the probability of constraint violation without quantifying the potential severity of the violation. To the best of our knowledge, the problem of interest has not been solved. To solve the problem, we propose and study a family of stochastic dynamic programs on an augmented state space. We prove that the optimal CVaR of a maximum random cost enjoys an equivalent representation in terms of the solutions to these dynamic programs under appropriate assumptions. For each dynamic program, we show the existence of an optimal policy that depends on the dynamics of an augmented state under the assumptions. In a numerical example, we illustrate how our safety analysis framework is useful for assessing the severity of combined sewer overflows under precipitation uncertainty.