Koopman-based Policy Iteration for Robust Optimal Control

TL;DR

This paper introduces a Koopman-based approach to solve robust optimal control problems involving adversaries, using data-driven policy iteration to approximate the value function via Koopman operator methods.

Contribution

It presents a novel Koopman-based formulation of the Hamilton-Jacobi-Issac equation and develops a data-driven policy iteration algorithm for robust control.

Findings

Successfully approximates the optimal value function using Koopman operator techniques.

Provides a new data-driven method for solving robust control problems.

Demonstrates effectiveness in approximating solutions to complex control problems.

Abstract

Classically, the optimal control problem in the presence of an adversary is formulated as a two-player zero-sum differential game or an $H_\infty$ control problem. The solution to these problems can be obtained by solving the Hamilton-Jacobi-Issac equation (HJIE). We provide a novel Koopman-based expression of the HJIE, where the solutions can be obtained through the approximation of the Koopman operator itself. In particular, we developed a data-driven and model based policy iteration algorithm for approximating the optimal value function using a finite-dimensional approximation of the Koopman operator and generator.