A Class of Convex Optimization-Based Recursive Algorithms for Identification of Stochastic Systems

TL;DR

This paper introduces a unified class of convex optimization-based recursive algorithms for stochastic system identification, demonstrating their strong consistency, robustness to outliers, and computational efficiency over existing methods.

Contribution

It develops a general framework of convex optimization criteria and recursive algorithms for stochastic system identification, extending classical estimators and proving their convergence.

Findings

Algorithms are strongly consistent with probability one.

Proposed methods are robust against outliers.

Recursive algorithms are more efficient online than kernel-based methods.

Abstract

Focusing on identification, this paper develops a class of convex optimization-based criteria and correspondingly the recursive algorithms to estimate the parameter vector $\theta^{*}$ of a stochastic dynamic system. Not only do the criteria include the classical least-squares estimator but also the $L_l=|\cdot|^l, l\geq 1$, the Huber, the Log-cosh, and the Quantile costs as special cases. First, we prove that the minimizers of the convex optimization-based criteria converge to $\theta^{*}$ with probability one. Second, the recursive algorithms are proposed to find the estimates, which minimize the convex optimization-based criteria, and it is shown that these estimates also converge to the true parameter vector with probability one. Numerical examples are given, justifying the performance of the proposed algorithms including the strong consistency of the estimates, the robustness against outliers in the observations, and higher efficiency in online computation compared with the kernel-based regularization method due to the recursive nature.