More Efficient Estimation for Logistic Regression with Optimal Subsample

TL;DR

This paper introduces an improved and computationally efficient subsampling method for logistic regression that enhances estimation accuracy and can operate effectively with limited memory, using Poisson subsampling and bias correction techniques.

Contribution

It develops a new estimator based on optimal subsampling probabilities and Poisson subsampling, improving efficiency and computational feasibility in large-scale logistic regression.

Findings

The new estimator outperforms existing methods in efficiency.

Poisson subsampling yields more efficient estimators at higher sampling rates.

The approach remains consistent even with an inconsistent pilot estimator.

Abstract

In this paper, we propose improved estimation method for logistic regression based on subsamples taken according the optimal subsampling probabilities developed in Wang et al. 2018 Both asymptotic results and numerical results show that the new estimator has a higher estimation efficiency. We also develop a new algorithm based on Poisson subsampling, which does not require to approximate the optimal subsampling probabilities all at once. This is computationally advantageous when available random-access memory is not enough to hold the full data. Interestingly, asymptotic distributions also show that Poisson subsampling produces a more efficient estimator if the sampling rate, the ratio of the subsample size to the full data sample size, does not converge to zero. We also obtain the unconditional asymptotic distribution for the estimator based on Poisson subsampling. The proposed approach requires to use a pilot estimator to correct biases of un-weighted estimators. We further show that even if the pilot estimator is inconsistent, the resulting estimators are still consistent and asymptotically normal if the model is correctly specified.