Optimizing Social Utility in Sequential Experiments

TL;DR

This paper proposes a sequential experimental protocol with partial subsidy to improve social utility in high-cost, high-stakes product development trials, demonstrating significant efficiency gains.

Contribution

It introduces a belief MDP-based framework for optimal sequential experimentation with subsidies, enabling efficient computation of strategies and social welfare maximization.

Findings

Social utility can be increased by over 35% with the proposed protocol.

Optimal strategies can be computed efficiently via dynamic programming.

The social utility function is piecewise linear and convex over subsidy levels.

Abstract

Regulatory approval of products in high-stakes domains such as drug development requires statistical evidence of safety and efficacy through large-scale randomized controlled trials. However, the high financial cost of these trials may deter developers who lack absolute certainty in their product's efficacy, ultimately stifling the development of `moonshot' products that could offer high social utility. To address this inefficiency, in this paper, we introduce a statistical protocol for experimentation where the product developer (the agent) conducts a randomized controlled trial sequentially and the regulator (the principal) partially subsidizes its cost. By modeling the protocol using a belief Markov decision process, we show that the agent's optimal strategy can be found efficiently using dynamic programming. Further, we show that the social utility is a piecewise linear and convex function over the subsidy level the principal selects, and thus the socially optimal subsidy can also be found efficiently using divide-and-conquer. Simulation experiments using publicly available data on antibiotic development and approval demonstrate that our statistical protocol can be used to increase social utility by more than $35$$\%$ relative to standard, non-sequential protocols.