Optimal Subsampling Algorithms for Big Data Regressions

TL;DR

This paper develops optimal subsampling algorithms for large-scale generalized linear models, providing theoretical guarantees and practical adaptive methods to efficiently approximate maximum likelihood estimators with massive data.

Contribution

It introduces a new adaptive two-step subsampling algorithm with proven asymptotic properties and optimality under A- and L-criteria for big data regressions.

Findings

The proposed methods achieve accurate estimations with reduced computational cost.

Theoretical results establish consistency and asymptotic normality of the estimators.

Numerical experiments demonstrate the effectiveness on simulated and real datasets.

Abstract

To fast approximate maximum likelihood estimators with massive data, this paper studies the Optimal Subsampling Method under the A-optimality Criterion (OSMAC) for generalized linear models. The consistency and asymptotic normality of the estimator from a general subsampling algorithm are established, and optimal subsampling probabilities under the A- and L-optimality criteria are derived. Furthermore, using Frobenius norm matrix concentration inequalities, finite sample properties of the subsample estimator based on optimal subsampling probabilities are also derived. Since the optimal subsampling probabilities depend on the full data estimate, an adaptive two-step algorithm is developed. Asymptotic normality and optimality of the estimator from this adaptive algorithm are established. The proposed methods are illustrated and evaluated through numerical experiments on simulated and real datasets.