Unbounded Dynamic Programming via the Q-Transform

TL;DR

This paper introduces a novel Q-transform-based method for solving unbounded dynamic programming problems, converting them into bounded problems to enable easier analysis and solution, applicable to a wide range of decision problems.

Contribution

The paper presents a new transformation technique that simplifies solving unbounded dynamic programs, broadening applicability and ease of use compared to existing methods.

Findings

Successfully converts unbounded problems into bounded ones

Applicable to many common decision problems

Simplifies the solution process for complex dynamic programs

Abstract

We propose a new approach to solving dynamic decision problems with unbounded rewards based on the transformations used in Q-learning. In our case, the objective of the transform is to convert an unbounded dynamic program into a bounded one. The approach is general enough to handle problems for which existing methods struggle, and yet simple relative to other techniques and accessible for applied work. We show by example that many common decision problems satisfy our conditions.