Genuinely Robust Inference for Clustered Data

TL;DR

This paper identifies limitations of traditional cluster-robust inference in large clusters, formalizes the issue, and introduces a new bootstrap method that ensures valid and reliable statistical inference across various data scenarios.

Contribution

It provides a necessary and sufficient condition for the validity of cluster inference and proposes a new bootstrap procedure that remains robust when conventional methods fail.

Findings

77% of empirical studies violate the validity condition

The new bootstrap method controls size across broad data classes

Using the method can significantly change statistical conclusions

Abstract

Conventional cluster-robust inference can be invalid when data contain clusters of unignorably large size. We formalize this issue by deriving a necessary and sufficient condition for its validity, and show that this condition is frequently violated in practice: specifications from 77% of empirical research articles in American Economic Review and Econometrica during 2020-2021 appear not to meet it. To address this limitation, we propose a genuinely robust inference procedure based on a new cluster score bootstrap. We establish its validity and size control across broad classes of data-generating processes where conventional methods break down. Simulation studies corroborate our theoretical findings, and empirical applications illustrate that employing the proposed method can substantially alter conventional statistical conclusions.