Treatment effect estimation with Multilevel Regression and Poststratification

TL;DR

This paper evaluates the use of Multilevel Regression and Poststratification (MRP) for causal inference in experimental settings, demonstrating its advantages over traditional methods in small-area treatment effect estimation.

Contribution

It extends MRP methodology to causal inference, incorporating non-census covariates, and compares its performance with standard methods through simulation studies.

Findings

MRP-style estimators have lower bias and variance in small-area treatment effect estimates.

MRP performs well even with treatment effect heterogeneity.

Incorporating non-census covariates enhances MRP's effectiveness.

Abstract

Multilevel regression and poststratification (MRP) is a flexible modeling technique that has been used in a broad range of small-area estimation problems. Traditionally, MRP studies have been focused on non-causal settings, where estimating a single population value using a nonrepresentative sample was of primary interest. In this manuscript, MRP-style estimators will be evaluated in an experimental causal inference setting. We simulate a large-scale randomized control trial with a stratified cluster sampling design, and compare traditional and nonparametric treatment effect estimation methods with MRP methodology. Using MRP-style estimators, treatment effect estimates for areas as small as 1.3$\%$ of the population have lower bias and variance than standard causal inference methods, even in the presence of treatment effect heterogeneity. The design of our simulation studies also requires us to build upon a MRP variant that allows for non-census covariates to be incorporated into poststratification.