A Generalizability Score for Aggregate Causal Effect

TL;DR

This paper introduces a generalizability score to improve the reliability of generalizing causal effects across populations, especially when covariate overlap is limited, by guiding target subpopulation selection.

Contribution

It proposes a novel generalizability score that helps select target subpopulations for causal effect generalization, reducing bias and variance issues.

Findings

The score improves subpopulation selection for causal generalization.

Simulation studies show the score's effectiveness in various scenarios.

Real data analysis confirms practical utility of the score.

Abstract

Scientists frequently generalize population level causal quantities such as average treatment effect from a source population to a target population. When the causal effects are heterogeneous, differences in subject characteristics between the source and target populations may make such a generalization difficult and unreliable. Reweighting or regression can be used to adjust for such differences when generalizing. However, these methods typically suffer from large variance if there is limited covariate distribution overlap between the two populations. We propose a generalizability score to address this issue. The score can be used as a yardstick to select target subpopulations for generalization. A simplified version of the score avoids using any outcome information and thus can prevent deliberate biases associated with inadvertent access to such information. Both simulation studies and real data analysis demonstrate convincing results for such selection.