Recovery of the optimal control value function in reproducing kernel Hilbert spaces from verification conditions

TL;DR

This paper introduces an RKHS-based framework for recovering the optimal value function in nonlinear control problems from verification conditions, ensuring convergence and linking to policy iteration.

Contribution

It develops a novel abstract recovery method in RKHS for approximating the value function from verification conditions, with proven convergence and practical implementation via policy iteration.

Findings

RKHS approximants converge to the true value function as collocation points become dense.

The method guarantees global convergence for analytic value functions.

Numerical experiments demonstrate the approach's effectiveness.

Abstract

Approximating the optimal value function $v^*$ for infinite-horizon, nonlinear, autonomous optimal control problems is both challenging and essential for synthesizing real-time optimal feedback. We develop an abstract optimal recovery framework in reproducing kernel Hilbert spaces (RKHS) for reconstructing unknown target functions from mixed equality and inequality functional constraints. Within this framework, the approximation of $v^*$ is cast as a collocation-type problem derived from verification conditions for optimality -- most prominently, the Hamilton-Jacobi-Bellman (HJB) equation -- that uniquely characterizes $v^*$. As the set of collocation points becomes dense in the ambient domain $\Omega$, we establish convergence of the RKHS approximants to $v^*$: globally on $\Omega$ in the RKHS norm when $v^*$ is analytic, and locally (in a neighborhood of the origin) in the RKHS norm when $v^*$ is bounded from above and below by quadratic functions. Furthermore, we show that a practical numerical realization of the abstract scheme reduces to the classical policy iteration algorithm. Numerical experiments support the effectiveness of the proposed approach.