Controlling Unknown Linear Dynamics with Almost Optimal Regret

TL;DR

This paper develops a control strategy for unknown linear dynamics that nearly minimizes regret without prior knowledge of the system parameter, achieving near-optimal performance for any real parameter.

Contribution

It introduces a control method that guarantees near-optimal regret bounds for any real unknown parameter, without requiring prior information.

Findings

Achieves regret within a (1+ε) factor of the optimal for any ε>0.

Applicable to any real-valued unknown system parameter.

Provides a universal control strategy with near-optimal regret bounds.

Abstract

Here and in a companion paper, we consider a simple control problem in which the underlying dynamics depend on a parameter $a$ that is unknown and must be learned. In this paper, we assume that $a$ can be any real number and we do not assume that we have a prior belief about $a$. We seek a control strategy that minimizes a quantity called the regret. Given any $\varepsilon>0$, we produce a strategy that minimizes the regret to within a multiplicative factor of $(1+\varepsilon)$.