A Simple Online Parameter Estimation Technique with Asymptotic Guarantees

TL;DR

This paper introduces a simple online parameter estimation method that operates efficiently with low memory, handles correlated data, and provides asymptotic guarantees, matching offline methods in accuracy.

Contribution

The paper presents a novel online estimation technique with asymptotic normality and covariance computation, suitable for low memory and correlated data scenarios.

Findings

Estimators are asymptotically normal under broad conditions.

The method achieves efficiency comparable to offline estimators.

Confidence intervals generated match offline counterparts.

Abstract

In many modern settings, data are acquired iteratively over time, rather than all at once. Such settings are known as online, as opposed to offline or batch. We introduce a simple technique for online parameter estimation, which can operate in low memory settings, settings where data are correlated, and only requires a single inspection of the available data at each time period. We show that the estimators---constructed via the technique---are asymptotically normal under generous assumptions, and present a technique for the online computation of the covariance matrices for such estimators. A set of numerical studies demonstrates that our estimators can be as efficient as their offline counterparts, and that our technique generates estimates and confidence intervals that match their offline counterparts in various parameter estimation settings.