Long run stochastic control problems with general discounting

TL;DR

This paper investigates long-term stochastic control problems with general discounting, establishing relationships between average reward and risk-sensitive reward, and providing asymptotic results for near-zero risk factors.

Contribution

It demonstrates that optimal strategies for average reward are also optimal under general discounting and analyzes the asymptotic behavior of risk-sensitive control as the risk factor approaches zero.

Findings

Optimal strategies for average reward are also optimal for general discounting.

Risk-sensitive reward functional is dominated by the average reward when risk factor is positive.

Near-zero risk factor, the optimal control for average reward approximates the risk-sensitive control.

Abstract

Controlled discrete time Markov processes are studied first with long run general discounting functional. It is shown that optimal strategies for average reward per unit time problem are also optimal for average generally discounting functional. Then long run risk sensitive reward functional with general discounting is considered. When risk factor is positive then optimal value of such reward functional is dominated by the reward functional corresponding to the long run risk sensitive control. In the case of negative risk factor we get an asymptotical result, which says that optimal average reward per unit time control is nearly optimal for long run risk sensitive reward functional with general discounting, assuming that risk factor is close to $0$. For this purpose we show in Appendix upper estimates for large deviations of weighted empirical measures, which are of independent interest.