Online Optimal Control with Linear Dynamics and Predictions: Algorithms and Regret Analysis

TL;DR

This paper introduces the Receding Horizon Gradient-based Control (RHGC) algorithm for online linear control with lookahead predictions, achieving near-optimal regret decay and demonstrating effectiveness through numerical tests.

Contribution

The paper proposes a novel online control algorithm that leverages finite lookahead predictions, with theoretical regret bounds close to fundamental limits, and validates its performance numerically.

Findings

Regret decays exponentially with lookahead window size.

RHGC nearly attains the fundamental regret limit.

Numerical tests confirm effectiveness for linear and nonlinear systems.

Abstract

This paper studies the online optimal control problem with time-varying convex stage costs for a time-invariant linear dynamical system, where a finite lookahead window of accurate predictions of the stage costs are available at each time. We design online algorithms, Receding Horizon Gradient-based Control (RHGC), that utilize the predictions through finite steps of gradient computations. We study the algorithm performance measured by dynamic regret: the online performance minus the optimal performance in hindsight. It is shown that the dynamic regret of RHGC decays exponentially with the size of the lookahead window. In addition, we provide a fundamental limit of the dynamic regret for any online algorithms by considering linear quadratic tracking problems. The regret upper bound of one RHGC method almost reaches the fundamental limit, demonstrating the effectiveness of the algorithm. Finally, we numerically test our algorithms for both linear and nonlinear systems to show the effectiveness and generality of our RHGC.