Online convex optimization for data-driven control of dynamical systems

TL;DR

This paper introduces a data-driven online convex optimization algorithm for controlling unknown linear dynamical systems with time-varying costs, achieving sublinear regret and noise estimation without requiring a system model.

Contribution

It presents a novel model-free control algorithm that handles unknown systems, noisy feedback, and time-varying costs using behavioral systems theory and persistent excitation.

Findings

Achieves sublinear regret in controlling unknown systems.

Effectively estimates measurement noise asymptotically.

Demonstrates applicability through thermal control simulation.

Abstract

We propose an algorithm based on online convex optimization for controlling discrete-time linear dynamical systems. The algorithm is data-driven, i.e., does not require a model of the system, and is able to handle a priori unknown and time-varying cost functions. To this end, we make use of a single persistently exciting input-output sequence of the system and results from behavioral systems theory which enable it to handle unknown linear time-invariant systems. Moreover, we consider noisy output feedback instead of full state measurements and allow general economic cost functions. Our analysis of the closed loop reveals that the algorithm is able to achieve sublinear regret, where the measurement noise only adds an additional constant term to the regret upper bound. In order to do so, we derive a data-driven characterization of the steady-state manifold of an unknown system. Moreover, our algorithm is able to asymptotically exactly estimate the measurement noise. The effectiveness and applicational aspects of the proposed method are illustrated by means of a detailed simulation example in thermal control.