Online Learning for Predictive Control with Provable Regret Guarantees

TL;DR

This paper develops online Model Predictive Control algorithms for unknown linear systems with time-varying costs, providing provable regret guarantees and demonstrating their effectiveness through numerical experiments.

Contribution

It introduces two novel online MPC algorithms with regret bounds for unknown systems, extending existing methods to more realistic settings.

Findings

CE-MPC achieves $ ext{O}(T^{2/3})$ regret under stability assumptions.

O-MPC also achieves $ ext{O}(T^{2/3})$ regret with relaxed assumptions.

Numerical studies confirm the algorithms' practical performance.

Abstract

We study the problem of online learning in predictive control of an unknown linear dynamical system with time varying cost functions which are unknown apriori. Specifically, we study the online learning problem where the control algorithm does not know the true system model and has only access to a fixed-length (that does not grow with the control horizon) preview of the future cost functions. The goal of the online algorithm is to minimize the dynamic regret, defined as the difference between the cumulative cost incurred by the algorithm and that of the best sequence of actions in hindsight. We propose two different online Model Predictive Control (MPC) algorithms to address this problem, namely Certainty Equivalence MPC (CE-MPC) algorithm and Optimistic MPC (O-MPC) algorithm. We show that under the standard stability assumption for the model estimate, the CE-MPC algorithm achieves $\mathcal{O}(T^{2/3})$ dynamic regret. We then extend this result to the setting where the stability assumption holds only for the true system model by proposing the O-MPC algorithm. We show that the O-MPC algorithm also achieves $\mathcal{O}(T^{2/3})$ dynamic regret, at the cost of some additional computation. We also present numerical studies to demonstrate the performance of our algorithm.