On the implementation of Approximate Randomization Tests in Linear Models with a Small Number of Clusters

TL;DR

This paper offers a comprehensive guide to implementing approximate randomization tests in linear models with few clusters, including new theoretical insights, practical algorithms, and applications.

Contribution

It introduces novel results on weighted score implementation, invariance properties, and convexity of confidence intervals, enhancing the methodology's robustness and usability.

Findings

Method applies to as few as five clusters

Test and intervals are invariant to studentization

Convexity of confidence intervals is established

Abstract

This paper provides a user's guide to the general theory of approximate randomization tests developed in Canay, Romano, and Shaikh (2017) when specialized to linear regressions with clustered data. An important feature of the methodology is that it applies to settings in which the number of clusters is small -- even as small as five. We provide a step-by-step algorithmic description of how to implement the test and construct confidence intervals for the parameter of interest. In doing so, we additionally present three novel results concerning the methodology: we show that the method admits an equivalent implementation based on weighted scores; we show the test and confidence intervals are invariant to whether the test statistic is studentized or not; and we prove convexity of the confidence intervals for scalar parameters. We also articulate the main requirements underlying the test, emphasizing in particular common pitfalls that researchers may encounter. Finally, we illustrate the use of the methodology with two applications that further illuminate these points. The companion {\tt R} and {\tt Stata} packages facilitate the implementation of the methodology and the replication of the empirical exercises.