Dynamic Optimal Choice When Rewards are Unbounded Below

TL;DR

This paper introduces a method to transform and solve dynamic decision problems with unbounded below rewards by converting them into bounded problems, enabling the use of contraction mapping for solution.

Contribution

It presents a novel transformation of the Bellman equation that allows solving certain unbounded reward problems as bounded ones using contraction mapping.

Findings

The transformation makes many common problems solvable with standard methods.

The approach is effective under specific conditions outlined in the paper.

Examples demonstrate the method's applicability to real decision problems.

Abstract

We propose a new approach to solving dynamic decision problems with rewards that are unbounded below. The approach involves transforming the Bellman equation in order to convert an unbounded problem into a bounded one. The major advantage is that, when the conditions stated below are satisfied, the transformed problem can be solved by iterating with a contraction mapping. While the method is not universal, we show by example that many common decision problems do satisfy our conditions.