Distributed Estimation and Inference for Semi-parametric Binary Response Models

TL;DR

This paper introduces novel distributed estimation methods for semi-parametric binary response models, improving computational efficiency and statistical accuracy through smoothing and iterative techniques, applicable to large, heterogeneous, and high-dimensional datasets.

Contribution

It proposes one-shot and multi-round smoothing-based divide-and-conquer estimators that relax computational constraints and achieve quadratic convergence in distributed semi-parametric binary response models.

Findings

Superlinear improvement of optimization error with iterative smoothing

Quadratic convergence up to the optimal statistical error rate

Extensions to heterogeneous and high-dimensional data scenarios

Abstract

The development of modern technology has enabled data collection of unprecedented size, which poses new challenges to many statistical estimation and inference problems. This paper studies the maximum score estimator of a semi-parametric binary choice model under a distributed computing environment without pre-specifying the noise distribution. An intuitive divide-and-conquer estimator is computationally expensive and restricted by a non-regular constraint on the number of machines, due to the highly non-smooth nature of the objective function. We propose (1) a one-shot divide-and-conquer estimator after smoothing the objective to relax the constraint, and (2) a multi-round estimator to completely remove the constraint via iterative smoothing. We specify an adaptive choice of kernel smoother with a sequentially shrinking bandwidth to achieve the superlinear improvement of the optimization error over the multiple iterations. The improved statistical accuracy per iteration is derived, and a quadratic convergence up to the optimal statistical error rate is established. We further provide two generalizations to handle the heterogeneity of datasets and high-dimensional problems where the parameter of interest is sparse.