Powering population health research: Considerations for plausible and actionable effect sizes

TL;DR

This paper discusses how to select realistic and actionable effect sizes for population health research, emphasizing that small effects can be valuable if interventions are broadly implemented and adequately powered studies are conducted.

Contribution

It provides guidance on choosing effect sizes for population health studies based on real intervention examples, highlighting that plausible effects are often smaller than traditional guidelines suggest.

Findings

Effect sizes depend on intervention, population, and outcomes.

Small effects can be impactful if interventions are widely implemented.

Large sample sizes are needed to detect small but meaningful effects.

Abstract

Evidence for Action (E4A), a signature program of the Robert Wood Johnson Foundation, funds investigator-initiated research on the impacts of social programs and policies on population health and health inequities. Across thousands of letters of intent and full proposals E4A has received since 2015, one of the most common methodological challenges faced by applicants is selecting realistic effect sizes to inform power and sample size calculations. E4A prioritizes health studies that are both (1) adequately powered to detect effect sizes that may reasonably be expected for the given intervention and (2) likely to achieve intervention effects sizes that, if demonstrated, correspond to actionable evidence for population health stakeholders. However, little guidance exists to inform the selection of effect sizes for population health research proposals. We draw on examples of five rigorously evaluated population health interventions. These examples illustrate considerations for selecting realistic and actionable effect sizes as inputs to power and sample size calculations for research proposals to study population health interventions. We show that plausible effects sizes for population health inteventions may be smaller than commonly cited guidelines suggest. Effect sizes achieved with population health interventions depend on the characteristics of the intervention, the target population, and the outcomes studied. Population health impact depends on the proportion of the population receiving the intervention. When adequately powered, even studies of interventions with small effect sizes can offer valuable evidence to inform population health if such interventions can be implemented broadly. Demonstrating the effectiveness of such interventions, however, requires large sample sizes.