Collaborative Design of Controlled Experiments in the Presence of Subject Covariates

TL;DR

This paper develops collaborative experimental design methods for multiple experiments with subject covariates, improving precision in treatment effect estimates by leveraging joint analysis and proposing two randomized algorithms with performance guarantees.

Contribution

It introduces a novel collaborative design framework for multiple experiments considering covariates and subject dependence, with two new algorithms for D-optimality solutions.

Findings

Algorithms outperform covariate-agnostic methods with many covariates

Collaborative design improves precision of treatment effect estimates

Proposed algorithms have theoretical performance guarantees

Abstract

We consider the optimal experimental design problem of allocating subjects to treatment or control when subjects participate in multiple, separate controlled experiments within a short time-frame and subject covariate information is available. Here, in addition to subject covariates, we consider the dependence among the responses coming from the subject's random effect across experiments. In this setting, the goal of the allocation is to provide precise estimates of treatment effects for each experiment. Deriving the precision matrix of the treatment effects and using D-optimality as our allocation criterion, we demonstrate the advantage of collaboratively designing and analyzing multiple experiments over traditional independent design and analysis, and propose two randomized algorithms to provide solutions to the D-optimality problem for collaborative design. The first algorithm decomposes the D-optimality problem into a sequence of subproblems, where each subproblem is a quadratic binary program that can be solved through a semi-definite relaxation based randomized algorithm with performance guarantees. The second algorithm involves solving a single semi-definite program, and randomly generating allocations for each experiment from the solution of this program. We showcase the performance of these algorithms through a simulation study, finding that our algorithms outperform covariate-agnostic methods when there are a large number of covariates.