Continuous-Time Robust Dynamic Programming

TL;DR

This paper introduces a new robust dynamic programming theory for continuous-time systems, enhancing the analysis of robustness and enabling the development of adaptive optimal control and reinforcement learning methods.

Contribution

The paper develops a novel robust dynamic programming framework for continuous-time systems, distinct from traditional DP, with applications in control and reinforcement learning.

Findings

Four illustrative applications in stochastic control and adaptive DP.

Three numerical examples from finance and engineering.

Potential extensions of the framework are discussed.

Abstract

This paper presents a new theory, known as robust dynamic pro- gramming, for a class of continuous-time dynamical systems. Different from traditional dynamic programming (DP) methods, this new theory serves as a fundamental tool to analyze the robustness of DP algorithms, and in par- ticular, to develop novel adaptive optimal control and reinforcement learning methods. In order to demonstrate the potential of this new framework, four illustrative applications in the fields of stochastic optimal control and adaptive DP are presented. Three numerical examples arising from both finance and engineering industries are also given, along with several possible extensions of the proposed framework.