Robustness to Modeling Errors in Risk-Sensitive Markov Decision Problems with Markov Risk Measures

TL;DR

This paper investigates the robustness of risk-sensitive Markov decision processes to modeling errors, providing conditions under which small parameter perturbations minimally impact optimal policies and value functions.

Contribution

It introduces sufficient conditions ensuring robustness of risk-sensitive MDPs to model perturbations, extending understanding of decision-making under model uncertainty.

Findings

Small model perturbations cause limited changes in optimal policies.

Robustness conditions are applicable to data-driven and preference-uncertain systems.

Implications for systems with changing noise distributions are discussed.

Abstract

We consider risk-sensitive Markov decision processes (MDPs), where the MDP model is influenced by a parameter which takes values in a compact metric space. We identify sufficient conditions under which small perturbations in the model parameters lead to small changes in the optimal value function and optimal policy. We further establish the robustness of the risk-sensitive optimal policies to modeling errors. Implications of the results for data-driven decision-making, decision-making with preference uncertainty, and systems with changing noise distributions are discussed.