Improving the Power of Economic Experiments Using Adaptive Designs

TL;DR

This paper demonstrates how adaptive, two-stage experimental designs borrowed from clinical trials can significantly enhance the statistical power of economic experiments without increasing sample sizes or costs.

Contribution

It introduces an application of adaptive design methods to economic experiments, improving their power while maintaining error control, supported by simulations and real data examples.

Findings

Power to reject hypotheses can be increased using adaptive designs.

Strong control of Type I error is maintained.

No increase in total sample size or costs is needed.

Abstract

An important issue for many economic experiments is how the experimenter can ensure sufficient power for rejecting one or more hypotheses. Here, we apply methods developed mainly within the area of clinical trials for testing multiple hypotheses simultaneously in adaptive, two-stage designs. Our main goal is to illustrate how this approach can be used to improve the power of economic experiments. Having briefly introduced the relevant theory, we perform a simulation study supported by the open source R package asd in order to evaluate the power of some different designs. The simulations show that the power to reject at least one hypothesis can be improved while still ensuring strong control of the overall Type I error probability, and without increasing the total sample size and thus the costs of the study. The derived designs are further illustrated by applying them to two different real-world data sets from experimental economics.