Online Algorithms for Recovery of Low-Rank Parameter Matrix in Non-stationary Stochastic Systems

TL;DR

This paper introduces an online two-stage algorithm for recovering low-rank matrices in non-stationary stochastic systems, combining RLS estimation with nuclear norm regularization to achieve accurate, adaptive, and provably consistent results.

Contribution

The paper proposes a novel online algorithm that combines RLS and weighted nuclear norm regularization for low-rank matrix recovery in non-stationary systems, with proven oracle properties.

Findings

The algorithm can identify the true rank with finite data.

It provides consistent estimates as data increases.

Asymptotic normality of estimates is established.

Abstract

This paper presents a two-stage online algorithm for recovery of low-rank parameter matrix in non-stationary stochastic systems. The first stage applies the recursive least squares (RLS) estimator combined with its singular value decomposition to estimate the unknown parameter matrix within the system, leveraging RLS for adaptability and SVD to reveal low-rank structure. The second stage introduces a weighted nuclear norm regularization criterion function, where adaptive weights derived from the first-stage enhance low-rank constraints. The regularization criterion admits an explicit and online computable solution, enabling efficient online updates when new data arrive without reprocessing historical data. Under the non-stationary and the non-persistent excitation conditions on the systems, the algorithm provably achieves: (i) the true rank of the unknown parameter matrix can be identified with a finite number of observations, (ii) the values of the matrix components can be consistently estimated as the number of observations increases, and (iii) the asymptotical normality of the algorithm is established as well. Such properties are termed oracle properties in the literature. Numerical simulations validate performance of the algorithm in estimation accuracy.