A Massive Data Framework for M-Estimators with Cubic-Rate

TL;DR

This paper develops a theoretical framework for divide and conquer methods applied to cubic-rate M-estimators in massive data settings, showing improved convergence and normality of aggregated estimators.

Contribution

It introduces a general theory for the asymptotic distribution of aggregated cubic-rate M-estimators, applicable to various estimators and demonstrating faster convergence.

Findings

Aggregated estimators achieve faster convergence rates.

Asymptotic normality of the aggregated estimators is established.

Simulation results validate the theoretical improvements.

Abstract

The divide and conquer method is a common strategy for handling massive data. In this article, we study the divide and conquer method for cubic-rate estimators under the massive data framework. We develop a general theory for establishing the asymptotic distribution of the aggregated M-estimators using a simple average. Under certain condition on the growing rate of the number of subgroups, the resulting aggregated estimators are shown to have faster convergence rate and asymptotic normal distribution, which are more tractable in both computation and inference than the original M-estimators based on pooled data. Our theory applies to a wide class of M-estimators with cube root convergence rate, including the location estimator, maximum score estimator and value search estimator. Empirical performance via simulations also validate our theoretical findings.