Risk-sensitive Markov control processes

TL;DR

This paper develops a comprehensive framework for risk-sensitive control in Markov processes, introducing new risk measures, a novel discount scheme, and stability conditions to solve infinite-horizon optimization problems.

Contribution

It generalizes risk measures for Markov control processes on Borel spaces, proposes a new discount scheme, and establishes stability conditions for average risk optimization.

Findings

Introduced a general risk framework for Markov control processes.

Proposed a new discount scheme consistent with existing literature.

Established stability conditions for average risk optimality.

Abstract

We introduce a general framework for measuring risk in the context of Markov control processes with risk maps on general Borel spaces that generalize known concepts of risk measures in mathematical finance, operations research and behavioral economics. Within the framework, applying weighted norm spaces to incorporate also unbounded costs, we study two types of infinite-horizon risk-sensitive criteria, discounted total risk and average risk, and solve the associated optimization problems by dynamic programming. For the discounted case, we propose a new discount scheme, which is different from the conventional form but consistent with the existing literature, while for the average risk criterion, we state Lyapunov-like stability conditions that generalize known conditions for Markov chains to ensure the existence of solutions to the optimality equation.