Online Tracking of a Predictable Drifting Parameter of a Time Series

TL;DR

This paper introduces an online algorithm for tracking a multidimensional, predictable, time-varying parameter in a time series, providing non-asymptotic error bounds and broad applicability across various models.

Contribution

It develops a general online tracking algorithm with theoretical error bounds, applicable to many stochastic approximation frameworks and models.

Findings

Provides uniform non-asymptotic error bounds for the tracking algorithm.

Demonstrates the construction of appropriate gain functions for different models.

Covers classical procedures like Robbins-Monro and Kiefer-Wolfowitz within a unified framework.

Abstract

We propose an online algorithm for tracking a multidimensional time-varying parameter of a time series, which is also allowed to be a predictable process with respect to the underlying time series. The algorithm is driven by a gain function. Under assumptions on the gain, we derive uniform non-asymptotic error bounds on the tracking algorithm in terms of chosen step size for the algorithm and the variation of the parameter of interest. We also outline how appropriate gain functions can be constructed. We give several examples of different variational setups for the parameter process where our result can be applied. The proposed approach covers many frameworks and models (including the classical Robbins-Monro and Kiefer-Wolfowitz procedures) where stochastic approximation algorithms comprise the main inference tool for the data analysis. We treat in some detail a couple of specific models.