Blackwell optimality in risk-sensitive stochastic control

TL;DR

This paper explores Blackwell optimality in risk-sensitive Markov Decision Processes, analyzing its properties, differences from risk-neutral cases, and its relation to stationarity, supported by illustrative examples.

Contribution

It introduces the concept of Blackwell optimality in risk-sensitive MDPs, examines its properties, and clarifies its relation to stationarity with illustrative examples.

Findings

Blackwell optimality can be characterized in risk-sensitive MDPs.

Risk-sensitive entropy affects the optimality and stationarity properties.

An example illustrates the structural differences between risk-sensitive and risk-neutral cases.

Abstract

In this paper, we consider a discrete-time Markov Decision Process (MDP) on a finite state-action space with a long-run risk-sensitive criterion used as the objective function. We discuss the concept of Blackwell optimality and comment on intricacies which arise when the risk-neutral expectation is replaced by the risk-sensitive entropy. Also, we show the relation between the Blackwell optimality and ultimate stationarity and provide an illustrative example that helps to better understand the structural difference between these two concepts.