Rate-Optimal Online Convex Optimization in Adaptive Linear Control

TL;DR

This paper introduces a computationally-efficient online control algorithm for unknown linear systems that achieves optimal regret rates in adversarial environments without requiring strong cost convexity assumptions.

Contribution

It presents the first efficient algorithm attaining optimal -regret in adaptive linear control without strong convexity assumptions on costs.

Findings

Achieves -regret in adversarial linear control.

Develops non-convex confidence bounds for online costs.

Introduces a novel regret minimization technique leveraging non-convex structure.

Abstract

We consider the problem of controlling an unknown linear dynamical system under adversarially changing convex costs and full feedback of both the state and cost function. We present the first computationally-efficient algorithm that attains an optimal $\smash{\sqrt{T}}$-regret rate compared to the best stabilizing linear controller in hindsight, while avoiding stringent assumptions on the costs such as strong convexity. Our approach is based on a careful design of non-convex lower confidence bounds for the online costs, and uses a novel technique for computationally-efficient regret minimization of these bounds that leverages their particular non-convex structure.