Efficient Online Linear Control with Stochastic Convex Costs and Unknown Dynamics

TL;DR

This paper introduces an efficient online control algorithm for unknown linear systems with stochastic convex costs, achieving optimal regret rates and improved computational efficiency through an optimism-based approach.

Contribution

It presents a novel, computationally efficient algorithm for online linear control with unknown dynamics, leveraging optimism to improve complexity and analysis.

Findings

Achieves  regret rate for unknown linear systems.

Uses optimism in the face of uncertainty paradigm.

Simplifies analysis and improves computational complexity.

Abstract

We consider the problem of controlling an unknown linear dynamical system under a stochastic convex cost and full feedback of both the state and cost function. We present a computationally efficient algorithm that attains an optimal $\sqrt{T}$ regret-rate compared to the best stabilizing linear controller in hindsight. In contrast to previous work, our algorithm is based on the Optimism in the Face of Uncertainty paradigm. This results in a substantially improved computational complexity and a simpler analysis.