Subsampled One-Step Estimation for Fast Statistical Inference

TL;DR

This paper introduces a subsampled one-step (SOS) estimator that significantly improves the efficiency of subsampling methods for large datasets, achieving near full-data accuracy with reduced computational cost.

Contribution

The paper proposes the SOS estimator that attains a faster convergence rate and asymptotic normality, bridging the gap between subsampling efficiency and full data estimation.

Findings

SOS estimator achieves convergence rate of max{n^{-1}, N^{-1/2}}

Establishes the asymptotic distribution of the SOS estimator, which can be non-normal

Numerical results show SOS is nearly as efficient as full data estimators while being computationally efficient.

Abstract

Subsampling is an effective approach to alleviate the computational burden associated with large-scale datasets. Nevertheless, existing subsampling estimators incur a substantial loss in estimation efficiency compared to estimators based on the full dataset. Specifically, the convergence rate of existing subsampling estimators is typically $n^{-1/2}$ rather than $N^{-1/2}$, where $n$ and $N$ denote the subsample and full data sizes, respectively. This paper proposes a subsampled one-step (SOS) method to mitigate the estimation efficiency loss utilizing the asymptotic expansions of the subsampling and full-data estimators. The resulting SOS estimator is computationally efficient and achieves a fast convergence rate of $\max\{n^{-1}, N^{-1/2}\}$ rather than $n^{-1/2}$. We establish the asymptotic distribution of the SOS estimator, which can be non-normal in general, and construct confidence intervals on top of the asymptotic distribution. Furthermore, we prove that the SOS estimator is asymptotically normal and equivalent to the full data-based estimator when $n / \sqrt{N} \to \infty$.Simulation studies and real data analyses were conducted to demonstrate the finite sample performance of the SOS estimator. Numerical results suggest that the SOS estimator is almost as computationally efficient as the uniform subsampling estimator while achieving similar estimation efficiency to the full data-based estimator.