Koopman Spectrum Nonlinear Regulators and Efficient Online Learning

TL;DR

This paper introduces a novel control paradigm based on Koopman spectrum cost for nonlinear systems, enabling more natural behaviors and broad dynamical characterizations, along with an efficient online learning algorithm.

Contribution

It proposes a new control framework using Koopman spectrum cost and develops a sample-efficient online learning algorithm with sub-linear regret.

Findings

Enables control of nonlinear systems with smooth, predictable motions.

Generalizes classical eigenstructure and pole assignment to nonlinear dynamics.

Provides an online learning algorithm with theoretical regret guarantees.

Abstract

Most modern reinforcement learning algorithms optimize a cumulative single-step cost along a trajectory. The optimized motions are often 'unnatural', representing, for example, behaviors with sudden accelerations that waste energy and lack predictability. In this work, we present a novel paradigm of controlling nonlinear systems via the minimization of the Koopman spectrum cost: a cost over the Koopman operator of the controlled dynamics. This induces a broader class of dynamical behaviors that evolve over stable manifolds such as nonlinear oscillators, closed loops, and smooth movements. We demonstrate that some dynamics characterizations that are not possible with a cumulative cost are feasible in this paradigm, which generalizes the classical eigenstructure and pole assignments to nonlinear decision making. Moreover, we present a sample efficient online learning algorithm for our problem that enjoys a sub-linear regret bound under some structural assumptions.