A subgroup-aware scoring approach to the study of effect modification in observational studies

TL;DR

This paper introduces a subgroup-aware scoring method to better detect effect modification in observational studies, addressing issues with existing methods that can be confounded by outliers.

Contribution

The authors propose a novel group M-statistic that improves effect modification detection by scoring matched pairs within subgroups, outperforming existing methods.

Findings

The new method demonstrates superior performance in extensive simulation settings.

Application to a malaria prevention study illustrates practical utility.

Addresses confounding issues caused by outliers in previous methods.

Abstract

Effect modification means the size of a treatment effect varies with an observed covariate. Generally speaking, a larger treatment effect with more stable error terms is less sensitive to bias. Thus, we might be able to conclude that a study is less sensitive to unmeasured bias by using these subgroups experiencing larger treatment effects. Lee et al. (2018) proposed the submax method that leverages the joint distribution of test statistics from subgroups to draw a firmer conclusion if effect modification occurs. However, one version of the submax method uses M-statistics as the test statistics and is implemented in the R package submax (Rosenbaum, 2017). The scaling factor in the M-statistics is computed using all observations combined across subgroups. We show that this combining can confuse effect modification with outliers. We propose a novel group M-statistic that scores the matched pairs in each subgroup to tackle the issue. We examine our novel scoring strategy in extensive settings to show the superior performance. The proposed method is applied to an observational study of the effect of a malaria prevention treatment in West Africa.