A variational formula for risk-sensitive reward

TL;DR

This paper derives a variational formula for the optimal growth rate in infinite horizon risk-sensitive control of Markov decision processes, extending classical eigenvalue formulas to a new maximization framework.

Contribution

It introduces a novel variational formula for the risk-sensitive control problem, generalizing Donsker and Varadhan's eigenvalue approach to a concave maximization setting.

Findings

Provides a new variational characterization of the optimal growth rate.

Extends classical eigenvalue formulas to risk-sensitive control.

Enables concave maximization approaches for the problem.

Abstract

We derive a variational formula for the optimal growth rate of reward in the infinite horizon risk-sensitive control problem for discrete time Markov decision processes with compact metric state and action spaces, extending a formula of Donsker and Varadhan for the Perron-Frobenius eigenvalue of a positive operator. This leads to a concave maximization formulation of the problem of determining this optimal growth rate.