Designing Persuasive Experiments

TL;DR

This paper introduces a framework for experimental design that aligns incentives between experimenters and regulators, optimizing social welfare while reducing sample sizes.

Contribution

It proposes a novel incentive-compatible framework that sets welfare thresholds and characterizes optimal sampling and stopping rules, improving efficiency in experimental design.

Findings

Sampling according to Neyman-allocation is always optimal under normal priors.

The framework reduces expected sample sizes by over 48% in a clinical-trial calibration.

It mitigates strategic Bayesian persuasion without requiring knowledge of private preferences.

Abstract

Incentives in experimental design are often misaligned: experimenters design and finance experiments to seek regulatory approval, while regulators seek to maximize social-welfare. We propose a framework to resolve this conflict, wherein regulators set a minimum expected welfare threshold, and experimenters optimize designs subject to this constraint. It requires no knowledge of experimenters' private preferences or costs and mitigates strategic Bayesian persuasion. Under normal priors, sampling according to the Neyman-allocation is always optimal, independent of the specific objectives. Furthermore, we characterize the optimal stopping-rule. In a numerical study calibrated to historical clinical-trial data, our framework reduces expected sample-sizes by over 48% relative to classical designs that attain the same social-welfare.