Time-inconsistent Risk-sensitive Equilibrium for Countable-stated Markov Decision Processes

TL;DR

This paper investigates time-inconsistent risk-sensitive control in countable-state Markov decision processes, establishing equilibrium strategies and their convergence as risk sensitivity diminishes.

Contribution

It introduces the concept of time-inconsistent equilibrium strategies for risk-sensitive MDPs and proves their existence and convergence in the limit case.

Findings

Existence of time-inconsistent equilibrium strategies.

Convergence of $ ext{e}$-equilibriums as $ ext{e} o 0^+$.

Validation of step-optimality in the control problem.

Abstract

This paper is devoted to solving a time-inconsistent risk-sensitive control problem with parameter $\e$ and its limit case ($\e\rightarrow0^+$) for countable-stated Markov decision processes (MDPs for short). Since the cost functional is time-inconsistent, it is impossible to find a global optimal strategy for both cases. Instead, for each case, we will prove the existence of time-inconstant equilibrium strategies which verify the so-called step-optimality. Moreover, we prove the convergence of $\e$-equilibriums and the corresponding value functions as $\e\rightarrow0^+$.