Optimal Adaptive Control of Linear Stochastic Systems with Quadratic Cost Function

TL;DR

This paper develops an adaptive control method for linear stochastic systems with unknown parameters, weakening traditional assumptions and ensuring optimality through a modified least squares algorithm with regularization and diminishing excitation.

Contribution

It introduces a novel adaptive control approach that guarantees optimality under weaker conditions by combining regularized least squares and diminishing excitation techniques.

Findings

Ensures uniform stabilizability and detectability of estimated models.

Achieves strong consistency of key parameter estimates.

Demonstrates optimality of certainty equivalence control with diminishing excitation.

Abstract

In this paper, we consider the adaptive linear quadratic Gaussian control problem, where both the linear transformation matrix of the state $A$ and the control gain matrix $B$ are unknown. The proposed adaptive optimal control only assumes that $(A, B)$ is stabilizable and $(A, Q^{1/2})$ is detectable, where $Q$ is the weighting matrix of the state in the quadratic cost function. This condition significantly weakens the classic assumptions used in the literature. To tackle this problem, a weighted least squares algorithm is modified by using random regularization method, which can ensure uniform stabilizability and uniform detectability of the family of estimated models. At the same time, a diminishing excitation is incorporated into the design of the proposed adaptive control to guarantee strong consistency of the desired components of the estimates. Finally, by utilizing this family of estimates, even if not all components of them converge to the true values, it is demonstrated that a certainty equivalence control with such a diminishing excitation is optimal for an ergodic quadratic cost function.