Regret Analysis of Distributed Online Control for LTI Systems with Adversarial Disturbances

TL;DR

This paper develops distributed online control algorithms for LTI systems under adversarial disturbances, providing regret bounds for both known and unknown dynamics scenarios, advancing the robustness and learning capabilities of multi-agent control systems.

Contribution

It introduces a fully distributed disturbance feedback controller with regret bounds for known dynamics and a novel explore-then-commit approach for unknown dynamics in adversarial settings.

Findings

Regret bound of $O(\sqrt{T}\log T)$ for known dynamics.

Regret bound of $O(T^{2/3} ext{poly}(\log T))$ for unknown dynamics.

Effective distributed control strategies under adversarial disturbances.

Abstract

This paper addresses the distributed online control problem over a network of linear time-invariant (LTI) systems (with possibly unknown dynamics) in the presence of adversarial perturbations. There exists a global network cost that is characterized by a time-varying convex function, which evolves in an adversarial manner and is sequentially and partially observed by local agents. The goal of each agent is to generate a control sequence that can compete with the best centralized control policy in hindsight, which has access to the global cost. This problem is formulated as a regret minimization. For the case of known dynamics, we propose a fully distributed disturbance feedback controller that guarantees a regret bound of $O(\sqrt{T}\log T)$, where $T$ is the time horizon. For the unknown dynamics case, we design a distributed explore-then-commit approach, where in the exploration phase all agents jointly learn the system dynamics, and in the learning phase our proposed control algorithm is applied using each agent system estimate. We establish a regret bound of $O(T^{2/3} \text{poly}(\log T))$ for this setting.