Evidence Aggregation for Treatment Choice

TL;DR

This paper develops an optimal evidence aggregation method for policy decisions using limited local data and external studies, applying a minimax-regret criterion to improve decision-making in public health and labor policies.

Contribution

It introduces a minimax-regret based aggregation rule for combining external evidence with limited local data, providing analytical insights and computational methods.

Findings

Effective policy decisions can be made using the proposed aggregation rule.

The rule performs well in minimizing welfare regret in case studies.

Application to labor and COVID-19 treatment demonstrates practical utility.

Abstract

Consider a planner who has limited knowledge of the policy's causal impact on a certain local population of interest due to a lack of data, but does have access to the publicized intervention studies performed for similar policies on different populations. How should the planner make use of and aggregate this existing evidence to make her policy decision? Following Manski (2020; Towards Credible Patient-Centered Meta-Analysis, \textit{Epidemiology}), we formulate the planner's problem as a statistical decision problem with a social welfare objective, and solve for an optimal aggregation rule under the minimax-regret criterion. We investigate the analytical properties, computational feasibility, and welfare regret performance of this rule. We apply the minimax regret decision rule to two settings: whether to enact an active labor market policy based on 14 randomized control trial studies; and whether to approve a drug (Remdesivir) for COVID-19 treatment using a meta-database of clinical trials.