Unbounded Markov Dynamic Programming with Weighted Supremum Norm Perov Contractions

TL;DR

This paper demonstrates how the Perov contraction theorem can be used to solve unbounded Markov dynamic programming problems by establishing weaker conditions for the existence of a unique solution, with applications to optimal savings.

Contribution

It introduces a novel approach combining weighted supremum norms with the Perov contraction theorem for unbounded reward functions in Markov decision processes.

Findings

Perov contraction theorem ensures unique fixed points under weaker conditions.

Spectral radius condition is nearly necessary for solutions in optimal savings.

Method extends classical contraction approaches to unbounded reward scenarios.

Abstract

This paper shows the usefulness of the Perov contraction theorem, which is a generalization of the classical Banach contraction theorem, for solving Markov dynamic programming problems. When the reward function is unbounded, combining an appropriate weighted supremum norm with the Perov contraction theorem yields a unique fixed point of the Bellman operator under weaker conditions than existing approaches. An application to the optimal savings problem shows that the average growth rate condition derived from the spectral radius of a certain nonnegative matrix is sufficient and almost necessary for obtaining a solution.