Risk-sensitive control of continuous time Markov chains

TL;DR

This paper investigates risk-sensitive control strategies for continuous-time Markov chains, providing characterizations of optimal controls for finite and infinite horizons, and introducing a policy iteration algorithm.

Contribution

It offers a comprehensive analysis of risk-sensitive control for Markov chains, including new characterizations and an algorithm for optimal control.

Findings

Characterization of value functions via HJB equations.

Existence of optimal stationary control under Lyapunov conditions.

Development of a policy iteration algorithm for control optimization.

Abstract

We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterise the value function via HJB equation and obtain an optimal Markov control. We do the same for infinite horizon discounted cost case. In the infinite horizon average cost case we establish the existence of an optimal stationary control under certain Lyapunov condition. We also develop a policy iteration algorithm for finding an optimal control.