Risk-averse formulations of Stochastic Optimal Control and Markov Decision Processes

TL;DR

This paper explores risk-averse and distributionally robust approaches to stochastic optimal control and MDPs, focusing on risk functionals like VaR, and provides conditions for optimal policies and sample complexity analysis.

Contribution

It introduces a framework for risk-averse modeling in SOC and MDPs, including conditions for optimal policies and analysis of sample complexity with VaR.

Findings

Derived necessary and sufficient conditions for non-randomized optimal policies.

Analyzed sample complexity of VaR-based optimization problems.

Discussed construction of nested risk functionals for risk modeling.

Abstract

The aim of this paper is to investigate risk-averse and distributionally robust modeling of Stochastic Optimal Control (SOC) and Markov Decision Process (MDP). We discuss construction of conditional nested risk functionals, a particular attention is given to the Value-at-Risk measure. Necessary and sufficient conditions for existence of non-randomized optimal policies in the framework of robust SOC and MDP are derived. We also investigate sample complexity of optimization problems involving the Value-at-Risk measure.