Robust Optimal Control Using Conditional RiskMappings in Infinite Horizon

TL;DR

This paper develops a risk-averse optimal control framework for infinite horizon Markov processes using conditional risk mappings, providing explicit solutions and approximations under mild conditions.

Contribution

It introduces a novel risk-averse formulation with explicit dynamic programming equations for unbounded costs in infinite horizon control problems.

Findings

Derived conditions for optimal strategy existence.

Provided explicit dynamic programming equations.

Demonstrated algorithms on investment and LQ regulator examples.

Abstract

We use one-step conditional risk mappings to formulate a risk averse version of a total cost problem on a controlled Markov process in discrete time infinite horizon. The nonnegative one step costs are assumed to be lower semi-continuous but not necessarily bounded. We derive the conditions for the existence of the optimal strategies and solve the problem explicitly by giving the robust dynamic programming equations under very mild conditions. We further give an $\epsilon$-optimal approximation to the solution and illustrate our algorithm in two examples of optimal investment and LQ regulator problems.