Principles of Statistical Inference in Online Problems

TL;DR

This paper introduces a new principle-driven approach for online long-run variance estimation that improves accuracy and efficiency, with applications to various online statistical procedures.

Contribution

It proposes a decomposition of kernel weights and inference principles that yield more efficient online variance estimators and enhances multiple online statistical methods.

Findings

Lower mean squared error compared to existing estimators

Effective in online quantile regression and change point detection

Reduces computational cost while improving statistical properties

Abstract

To investigate a dilemma of statistical and computational efficiency faced by long-run variance estimators, we propose a decomposition of kernel weights in a quadratic form and some online inference principles. These proposals allow us to characterize efficient online long-run variance estimators. Our asymptotic theory and simulations show that this principle-driven approach leads to online estimators with a uniformly lower mean squared error than all existing works. We also discuss practical enhancements such as mini-batch and automatic updates to handle fast streaming data and optimal parameters tuning. Beyond variance estimation, we consider the proposals in the context of online quantile regression, online change point detection, Markov chain Monte Carlo convergence diagnosis, and stochastic approximation. Substantial improvements in computational cost and finite-sample statistical properties are observed when we apply our principle-driven variance estimator to original and modified inference procedures.