Steady-state Based Approach to Online Non-stochastic Control

TL;DR

This paper introduces an online control algorithm with $\\mathcal{O}(\sqrt{T})$ regret against a broader set of steady-states achievable by affine controllers, improving performance guarantees in adversarial settings.

Contribution

It extends existing online control algorithms to achieve regret bounds against a richer benchmark set involving affine controllers, with a novel non-convex optimization approach.

Findings

Achieves $\mathcal{O}(\sqrt{T})$ regret against affine controller steady-states.

Uses a combination of Follow-The-Perturbed-Leader and batching for stability.

Numerical experiments show lower total cost with similar computational effort.

Abstract

We study the problem of online non-stochastic control (ONC), which is the control of a linear system under adversarial disturbances and adversarial cost functions, with the aim of minimizing the total cost incurred. A recent line of literature in ONC develops algorithms that enjoy sublinear regret with respect to a benchmark based on the set of steady-states that are attainable by a constant input. In this work, we extend this research direction by giving an algorithm that enjoys $\mathcal{O}(\sqrt{T})$ regret with respect to a richer benchmark set, namely the set of steady-states attainable under an \emph{affine controller}. Since this benchmark substantially broadens the comparison class, it provides significantly stronger performance guarantees. Our proposed algorithm combines a Follow-The-Perturbed-Leader-style online non-convex optimization approach with a batching method that maintains stability despite changing policies. Although our proposed algorithm requires solving non-convex subproblems, we show that an approximate solution to this subproblem is sufficient to ensure $\mathcal{O}(\sqrt{T})$ regret. Furthermore, numerical experiments show that our algorithm enjoys lower total cost and similar computation to existing methods in certain settings.