Simultaneous System Identification and Model Predictive Control with No Dynamic Regret

TL;DR

This paper introduces an algorithm for simultaneous system identification and model predictive control of nonlinear systems, with guarantees of near-optimality and sublinear dynamic regret, applicable to systems with unknown, adaptive disturbances.

Contribution

The paper presents a self-supervised algorithm that combines system identification with model predictive control, providing finite-time guarantees and handling unknown, adaptive disturbances modeled in reproducing kernel Hilbert spaces.

Findings

Algorithm achieves sublinear dynamic regret.

Validated on hardware and simulations with unmodeled disturbances.

Successfully controls systems with unknown, adaptive dynamics.

Abstract

We provide an algorithm for the simultaneous system identification and model predictive control of nonlinear systems. The algorithm has finite-time near-optimality guarantees and asymptotically converges to the optimal (non-causal) controller. Particularly, the algorithm enjoys sublinear dynamic regret, defined herein as the suboptimality against an optimal clairvoyant controller that knows how the unknown disturbances and system dynamics will adapt to its actions. The algorithm is self-supervised and applies to control-affine systems with unknown dynamics and disturbances that can be expressed in reproducing kernel Hilbert spaces. Such spaces can model external disturbances and modeling errors that can even be adaptive to the system's state and control input. For example, they can model wind and wave disturbances to aerial and marine vehicles, or inaccurate model parameters such as inertia of mechanical systems. The algorithm first generates random Fourier features that are used to approximate the unknown dynamics or disturbances. Then, it employs model predictive control based on the current learned model of the unknown dynamics (or disturbances). The model of the unknown dynamics is updated online using least squares based on the data collected while controlling the system. We validate our algorithm in both hardware experiments and physics-based simulations. The simulations include (i) a cart-pole aiming to maintain the pole upright despite inaccurate model parameters, and (ii) a quadrotor aiming to track reference trajectories despite unmodeled aerodynamic drag effects. The hardware experiments include a quadrotor aiming to track a circular trajectory despite unmodeled aerodynamic drag effects, ground effects, and wind disturbances.