Adaptive debiased SGD in high-dimensional GLMs with streaming data

TL;DR

This paper presents an adaptive online inference method for high-dimensional generalized linear models that updates estimates in real-time with low computational and storage costs, using a novel adaptive stochastic gradient descent and debiasing technique.

Contribution

It introduces an adaptive stochastic gradient descent algorithm with an online debiasing procedure for high-dimensional GLMs, enabling efficient real-time inference in streaming data.

Findings

Establishes asymptotic normality of the ADL estimator.

Demonstrates computational efficiency through simulations.

Validates the method with a real spam email classification dataset.

Abstract

Online statistical inference facilitates real-time analysis of sequentially collected data, making it different from traditional methods that rely on static datasets. This paper introduces a novel approach to online inference in high-dimensional generalized linear models, where we update regression coefficient estimates and their standard errors upon each new data arrival. In contrast to existing methods that either require full dataset access or large-dimensional summary statistics storage, our method operates in a single-pass mode, significantly reducing both time and space complexity. The core of our methodological innovation lies in an adaptive stochastic gradient descent algorithm tailored for dynamic objective functions, coupled with a novel online debiasing procedure. This allows us to maintain low-dimensional summary statistics while effectively controlling the optimization error introduced by the dynamically changing loss functions. We establish the asymptotic normality of our proposed Adaptive Debiased Lasso (ADL) estimator. We conduct extensive simulation experiments to show the statistical validity and computational efficiency of our ADL estimator across various settings. Its computational efficiency is further demonstrated via a real data application to the spam email classification.