Dynamic Programming Principle and Hamilton-Jacobi-Bellman Equation for Optimal Control Problems with Uncertainty

TL;DR

This paper establishes a connection between the value function of an uncertain optimal control problem and the Hamilton-Jacobi-Bellman equation using a Hilbert space formulation, proving uniqueness and existence of solutions.

Contribution

It introduces a novel infinite-dimensional framework for uncertain control problems and proves the value function as the unique solution to the HJB equation.

Findings

Value function is the unique lower semi-continuous proximal solution of the HJB equation.

Framework handles uncertainties in dynamics, costs, and initial conditions.

Uses invariance properties and dynamic programming principle in an infinite-dimensional setting.

Abstract

We study the properties of the value function associated with an optimal control problem with uncertainties, known as average or Riemann-Stieltjes problem. Uncertainties are assumed to belong to a compact metric probability space, and appear in the dynamics, in the terminal cost and in the initial condition, which yield an infinite-dimensional formulation. By stating the problem as an evolution equation in a Hilbert space, we show that the value function is the unique lower semi-continuous proximal solution of the Hamilton-Jacobi-Bellman (HJB) equation. Our approach relies on invariance properties and the dynamic programming principle.