Finite Time Regret Bounds for Minimum Variance Control of Autoregressive Systems with Exogenous Inputs

TL;DR

This paper introduces a new adaptive minimum variance control method with provable finite-time regret bounds for ARX systems, improving initial transient performance and providing theoretical guarantees.

Contribution

It proposes the PIECE controller with exploration, offering the first finite-time regret bounds for adaptive minimum variance control of ARX systems.

Findings

PIECE achieves $C \, \log T$ regret bound with bounded noise.

PIECE achieves $C \log^2 T$ regret bound with sub-Gaussian noise.

Simulation shows PIECE outperforms existing methods in early learning stages.

Abstract

Minimum variance controllers have been employed in a wide-range of industrial applications. A key challenge experienced by many adaptive controllers is their poor empirical performance in the initial stages of learning. In this paper, we address the problem of initializing them so that they provide acceptable transients, and also provide an accompanying finite-time regret analysis, for adaptive minimum variance control of an auto-regressive system with exogenous inputs (ARX). Following [3], we consider a modified version of the Certainty Equivalence (CE) adaptive controller, which we call PIECE, that utilizes probing inputs for exploration. We show that it has a $C \log T$ bound on the regret after $T$ time-steps for bounded noise, and $C\log^2 T$ in the case of sub-Gaussian noise. The simulation results demonstrate the advantage of PIECE over the algorithm proposed in [3] as well as the standard Certainty Equivalence controller especially in the initial learning phase. To the best of our knowledge, this is the first work that provides finite-time regret bounds for an adaptive minimum variance controller.