Partially Pooled Propensity Score Models for Average Treatment Effect Estimation with Multilevel Data

TL;DR

This paper introduces a partially pooled propensity score approach for estimating average treatment effects in multilevel data with clustered individuals, addressing unmeasured confounders and small cluster challenges.

Contribution

It proposes a novel clustering-based propensity score estimation method that reduces bias in multilevel observational studies with many small clusters.

Findings

Grouping clusters by similar treatment prevalence improves bias reduction.

The method performs well in numerical experiments and theoretical analysis.

Application to preschool program data demonstrates practical utility.

Abstract

Causal inference analyses often use existing observational data, which in many cases has some clustering of individuals. In this paper we discuss propensity score weighting methods in a multilevel setting where within clusters individuals share unmeasured confounders that are related to treatment assignment and the potential outcomes. We focus in particular on settings where models with fixed cluster effects are either not feasible or not useful due to the presence of a large number of small clusters. We found, both through numerical experiments and theoretical derivations, that a strategy of grouping clusters with similar treatment prevalence and estimating propensity scores within such cluster groups is effective in reducing bias from unmeasured cluster-level covariates under mild conditions on the outcome model. We apply our proposed method in evaluating the effectiveness of center-based pre-school program participation on children's achievement at kindergarten, using the Early Childhood Longitudinal Study, Kindergarten data.