When Can We Trust Cluster-Robust Inference?

TL;DR

This paper reviews various methods for cluster-robust inference in regression models, highlighting their strengths and limitations, and suggests using multiple procedures to assess the reliability of statistical inferences.

Contribution

It provides a comprehensive overview of different variance estimators and inference methods, guiding practitioners on assessing their trustworthiness in specific datasets.

Findings

Some methods outperform others in certain scenarios

Using multiple procedures improves confidence in inference reliability

No single method is universally reliable for all cases

Abstract

It is common when using cross-section or panel data to assign each observation to a cluster and allow for arbitrary patterns of heteroskedasticity and correlation within clusters. For regression models, there are many ways to make cluster-robust inferences. A number of different variance matrix estimators can be used. Hypothesis tests and confidence intervals can then be based on several alternative analytic or bootstrap distributions. Some methods typically perform much better than others, but no method yields reliable inferences in every case. Thus it can be hard to know which $P$ values and confidence intervals to trust. Nevertheless, by using a number of procedures to assess the reliability of various inferential methods for a specific model and dataset, we can often obtain results in which we may be reasonably confident.