Kernel-Based Optimal Control: An Infinitesimal Generator Approach

TL;DR

This paper introduces a kernel-based, operator-theoretic method for data-driven optimal control of nonlinear stochastic systems, utilizing infinitesimal generators within reproducing kernel Hilbert spaces to improve control solutions.

Contribution

It develops a novel framework that directly learns the infinitesimal generator of controlled stochastic systems in an infinite-dimensional space, integrating with Hamilton-Jacobi-Bellman recursions.

Findings

Effective in synthetic differential equations

Outperforms classical nonlinear programming methods

Applicable to simulated robotic systems

Abstract

This paper presents a novel operator-theoretic approach for optimal control of nonlinear stochastic systems within reproducing kernel Hilbert spaces. Our learning framework leverages data samples of system dynamics and stage cost functions, with only control penalties and constraints provided. The proposed method directly learns the infinitesimal generator of a controlled stochastic diffusion in an infinite-dimensional hypothesis space. We demonstrate that our approach seamlessly integrates with modern convex operator-theoretic Hamilton-Jacobi-Bellman recursions, enabling a data-driven solution to the optimal control problems. Furthermore, our learning framework includes nonparametric estimators for uncontrolled infinitesimal generators as a special case. Numerical experiments, ranging from synthetic differential equations to simulated robotic systems, showcase the advantages of our approach compared to both modern data-driven and classical nonlinear programming methods for optimal control.