Data-driven End-to-end Learning of Pole Placement Control for Nonlinear Dynamics via Koopman Invariant Subspaces

TL;DR

This paper introduces a data-driven, end-to-end reinforcement learning approach for controlling nonlinear dynamical systems by manipulating Koopman operator eigenvalues to achieve desired frequency and convergence properties.

Contribution

It develops a differentiable policy network that learns Koopman invariant subspaces and performs pole placement, enabling effective control of black-box nonlinear systems.

Findings

Outperforms model-free reinforcement learning in control tasks.

Achieves better convergence rates than traditional model-based control.

Demonstrates effectiveness on nonlinear dynamical systems.

Abstract

We propose a data-driven method for controlling the frequency and convergence rate of black-box nonlinear dynamical systems based on the Koopman operator theory. With the proposed method, a policy network is trained such that the eigenvalues of a Koopman operator of controlled dynamics are close to the target eigenvalues. The policy network consists of a neural network to find a Koopman invariant subspace, and a pole placement module to adjust the eigenvalues of the Koopman operator. Since the policy network is differentiable, we can train it in an end-to-end fashion using reinforcement learning. We demonstrate that the proposed method achieves better performance than model-free reinforcement learning and model-based control with system identification.