A Generalization of Bellman's Equation with Application to Path Planning, Obstacle Avoidance and Invariant Set Estimation

TL;DR

This paper generalizes Bellman's equation to handle a broader class of multi-stage optimization problems with monotonically backward separable costs, enabling efficient solutions for path planning and invariant set estimation.

Contribution

It introduces a necessary and sufficient condition based on the generalized Bellman's equation for optimality in these problems, extending beyond additive costs.

Findings

Efficient computation of optimal paths for Dubin's car with obstacle avoidance.

Determination of maximal invariant sets for discrete-time systems.

Generalization of Bellman's equation applicable to a wider class of MSOPs.

Abstract

The standard Dynamic Programming (DP) formulation can be used to solve Multi-Stage Optimization Problems (MSOP's) with additively separable objective functions. In this paper we consider a larger class of MSOP's with monotonically backward separable objective functions; additively separable functions being a special case of monotonically backward separable functions. We propose a necessary and sufficient condition, utilizing a generalization of Bellman's equation, for a solution of a MSOP, with a monotonically backward separable cost function, to be optimal. Moreover, we show that this proposed condition can be used to efficiently compute optimal solutions for two important MSOP's; the optimal path for Dubin's car with obstacle avoidance, and the maximal invariant set for discrete time systems.