On Distributional Discrepancy for Experimental Design with General Assignment Probabilities

TL;DR

This paper explores the challenge of experimental design in randomized controlled trials with unequal assignment probabilities, introducing a new algorithm that improves estimation accuracy and analyzing its computational complexity.

Contribution

It proves the NP-hardness of approximating the distributional discrepancy minimization problem and introduces a novel MWU algorithm that enhances RCT design with unequal probabilities.

Findings

MWU algorithm outperforms previous methods in simulations

Reduces worst-case mean squared error in treatment effect estimation

Establishes NP-hardness of the approximation problem

Abstract

We investigate experimental design for randomized controlled trials (RCTs) with both equal and unequal treatment-control assignment probabilities. Our work makes progress on the connection between the distributional discrepancy minimization (DDM) problem introduced by Harshaw et al. (2024) and the design of RCTs. We make two main contributions: First, we prove that approximating the optimal solution of the DDM problem within a certain constant error is NP-hard. Second, we introduce a new Multiplicative Weights Update (MWU) algorithm for the DDM problem, which improves the Gram-Schmidt walk algorithm used by Harshaw et al. (2024) when assignment probabilities are unequal. Building on the framework of Harshaw et al. (2024) and our MWU algorithm, we then develop the MWU design, which reduces the worst-case mean squared error in estimating the average treatment effect. Finally, we present a comprehensive simulation study comparing our design with commonly used designs.