Robustness to missing data: breakdown point analysis

TL;DR

This paper introduces a methodology to measure the robustness of econometric results against missing data by defining and estimating a breakdown point, applicable to GMM models and demonstrated through simulations and RCTs.

Contribution

It proposes a novel breakdown point measure for robustness analysis with an estimator that is root-n consistent and normal, applicable to GMM-based models.

Findings

The estimator performs well in finite samples for averages, linear, and logistic regressions.

The methodology effectively quantifies robustness of conclusions in the presence of missing data.

Application to RCTs illustrates practical utility in real-world scenarios.

Abstract

Missing data is pervasive in econometric applications, and rarely is it plausible that the data are missing (completely) at random. This paper proposes a methodology for studying the robustness of results drawn from incomplete datasets. Selection is measured as the divergence from the distribution of complete observations to the distribution of incomplete observations. The breakdown point is defined as the minimal amount of selection needed to overturn a given result. Reporting point estimates and lower confidence intervals of the breakdown point is a simple, concise way to communicate the robustness of a result. An estimator of the breakdown point is proposed and shown root-n consistent and asymptotically normal. This estimator can be applied directly to conclusions drawn from any model identified with the generalized method of moments (GMM) that satisfies mild assumptions. Simulations demonstrate the finite sample performance of the breakdown point estimator on averages, linear regression, and logistic regression. The methodology is illustrated by estimating the breakdown point of conclusions drawn from several randomized controlled trails suffering from missing data due to attrition.