Partial Identification of Treatment Effects for Generalizability

TL;DR

This paper develops partial identification methods to estimate bounds on treatment effects for generalizing from nonrandom samples, using weaker assumptions than traditional point estimate approaches, demonstrated with an educational cluster trial.

Contribution

It extends partial identification techniques to causal generalization, deriving bounds under minimal assumptions and incorporating population data to improve inference.

Findings

Bounds can rule out large treatment effects under certain assumptions.

Interval estimates are consistent with experimental point estimates.

Partial identification offers a viable alternative when strong ignorability assumptions are questionable.

Abstract

Recent methods to improve generalizations from nonrandom samples typically invoke assumptions such as the strong ignorability of sample selection that are often controversial in practice to derive point estimates. Rather than focus on the point estimate based inferences, this article considers inferences on partially identified estimates from fewer and weaker assumptions. We extend partial identification methods to causal generalization with nonrandom samples by using a cluster randomized trial in education. Bounds on the population average treatment effect are derived under four cases, two under no assumptions on the data, and two that assume bounded sample variation and monotonicity of response. This approach is amenable to incorporating population data frames to tighten bounds on the population average treatment effect. Under the assumptions of bounded sample variation and monotonicity, the interval estimates of the average treatment effect provide sufficiently informative bounds to rule out large treatment effects, which are consistent with the point estimates from the experimental study. This illustrates that partial identification methods can provide an alternative perspective to causal generalization in the absence of strong ignorability of sample selection.