Convergence of The Relative Value Iteration for the Ergodic Control Problem of Nondegenerate Diffusions under Near-Monotone Costs

TL;DR

This paper proves that the relative value iteration algorithm converges to the solution of the Hamilton-Jacobi-Bellman equation for ergodic control of nondegenerate diffusions with near-monotone costs, ensuring stability of the method.

Contribution

It establishes the convergence of the relative value iteration for a class of ergodic control problems involving nondegenerate diffusions and near-monotone costs, under broad conditions.

Findings

The quasilinear parabolic Cauchy problem stabilizes for all bounded initial conditions.

The solution converges to the HJB equation's solution, confirming the algorithm's validity.

Convergence holds in the setting of nondegenerate diffusions with near-monotone costs.

Abstract

We study the relative value iteration for the ergodic control problem under a near-monotone running cost structure for a nondegenerate diffusion controlled through its drift. This algorithm takes the form of a quasilinear parabolic Cauchy initial value problem in $\RR^{d}$. We show that this Cauchy problem stabilizes, or in other words, that the solution of the quasilinear parabolic equation converges for every bounded initial condition in $\Cc^{2}(\RR^{d})$ to the solution of the Hamilton--Jacobi--Bellman (HJB) equation associated with the ergodic control problem.