Adaptive discovery of effect modification in matched observational studies

TL;DR

This paper introduces a statistically rigorous method for discovering effect modification in matched observational studies, controlling false discoveries and accounting for unmeasured confounding.

Contribution

It presents a novel finite-sample valid procedure that identifies interpretable subgroups with controlled FDR, leveraging multiple controls and sensitivity models.

Findings

Method effectively controls subgroup-level FDR in simulations.

Improves power by using multiple matched controls.

Successfully applied to study economic returns to college.

Abstract

Understanding effect modification -- how treatment effects vary across subpopulations -- is practically important in observational studies, as it helps identify which subgroups are likely to benefit from a given treatment. In this paper, we study the discovery of effect modification in matched observational studies, where each treated unit may be matched to multiple controls. We develop a finite-sample valid procedure for identifying and selecting covariate-interpretable subgroups, with exact control of the subgroup-level false discovery rate (FDR). Our method explicitly accounts for unmeasured confounding via sensitivity models, and leverages multiple matched controls to improve statistical power. We demonstrate the favorable performance of our method relative to baseline methods through extensive simulation studies and a real-world application to the economic returns to college education.