Post-treatment problems: What can we say about the effect of a treatment among sub-groups who (would) respond in some way?

TL;DR

This paper introduces the TRACE method to estimate treatment effects among subgroups defined by their response to treatment, addressing biases from conditioning on post-treatment variables.

Contribution

It proposes a new causal inference framework that identifies and bounds effects among groups characterized by their potential responses to treatment.

Findings

Demonstrates the approach with three real-world examples.

Provides bounds for the effect among non-reactive groups.

Addresses biases from conditioning on post-treatment variables.

Abstract

Investigators are often interested in how a treatment affects an outcome for units responding to treatment in a certain way. We may wish to know the effect among units that, for example, meaningfully implemented an intervention, passed an attention check, or demonstrated some important mechanistic response. Simply conditioning on the observed value of the post-treatment variable introduces problematic biases. Further, the identification assumptions required of several existing strategies are often indefensible. We propose the Treatment Reactive Average Causal Effect (TRACE), which we define as the total effect of treatment in the group that, if treated, would realize a particular value of the relevant post-treatment variable. By reasoning about the effect among the "non-reactive" group, we can identify and estimate the range of plausible values for the TRACE. We demonstrate the use of this approach with three examples: (i) learning the effect of police-perceived race on police violence during traffic stops, a case where point identification may be possible; (ii) estimating effects of a community-policing intervention in Liberia, in communities that meaningfully implemented it, and (iii) studying how in-person canvassing affects support for transgender rights, among participants for whom the intervention would result in more positive feelings towards transgender people.