Selecting the Most Effective Nudge: Evidence from a Large-Scale Experiment on Immunization

TL;DR

This paper introduces Treatment Variant Aggregation (TVA), a new method for selecting effective policy combinations from large factorial experiments, demonstrated through a large-scale immunization trial in India.

Contribution

The paper develops TVA, a novel technique for efficiently identifying effective policy variants in large factorial designs, with application to immunization demand stimulation.

Findings

The optimal policy increased immunizations by 44%.

A cost-effective policy increased immunizations per dollar by 9.1%.

TVA effectively prunes ineffective policy variants.

Abstract

Policymakers often choose a policy bundle that is a combination of different interventions in different dosages. We develop a new technique -- treatment variant aggregation (TVA) -- to select a policy from a large factorial design. TVA pools together policy variants that are not meaningfully different and prunes those deemed ineffective. This allows us to restrict attention to aggregated policy variants, consistently estimate their effects on the outcome, and estimate the best policy effect adjusting for the winner's curse. We apply TVA to a large randomized controlled trial that tests interventions to stimulate demand for immunization in Haryana, India. The policies under consideration include reminders, incentives, and local ambassadors for community mobilization. Cross-randomizing these interventions, with different dosages or types of each intervention, yields 75 combinations. The policy with the largest impact (which combines incentives, ambassadors who are information hubs, and reminders) increases the number of immunizations by 44% relative to the status quo. The most cost-effective policy (information hubs, ambassadors, and SMS reminders but no incentives) increases the number of immunizations per dollar by 9.1% relative to status quo.