On the Fundamental Limit of the Stochastic Gradient Identification Algorithm Under Non-Persistent Excitation

TL;DR

This paper proves that stochastic gradient algorithms achieve strong consistency under non-persistent excitation for a wider range of conditions than previously known, nearly resolving a long-standing conjecture.

Contribution

It establishes strong consistency of the stochastic gradient method for all \\alpha in [0, 1), improving upon prior results limited to \\alpha \\le 1/3 and nearly resolving a four-decade-old conjecture.

Findings

Strong consistency holds for 0 \\le \\alpha < 1.

New algebraic framework yields sharper matrix norm bounds.

Nearly resolves the Chen--Guo conjecture for the entire range 1/3 < \\alpha < 1.

Abstract

Stochastic gradient (SG) methods are fundamental to system identification and machine learning, enabling online parameter estimation in large-scale and streaming-data settings. As a classical identification method, the SG algorithm has been extensively studied for decades. Under non-persistent excitation, the strongest currently available convergence result assumes that the condition number of the Fisher information matrix is \(O((\log r_n)^\alpha)\), where \(r_n = 1 + \sum_{i=1}^n \|\varphi_i\|^2\). Existing theory establishes strong consistency when \(\alpha \le 1/3\), whereas the same condition with \(\alpha > 1\) is insufficient to guarantee strong consistency. We prove that strong consistency holds throughout the range \(0 \le \alpha < 1\). The proof is based on a new algebraic framework that yields substantially sharper matrix norm bounds. This result nearly resolves the four-decade-old Chen--Guo conjecture by establishing strong consistency throughout the previously open range \(1/3 < \alpha < 1\).