Minimax designs for causal effects in temporal experiments with treatment habituation

TL;DR

This paper introduces minimax optimal randomized designs for estimating causal effects in temporal experiments where treatment habituation occurs, using a non-parametric causal inference framework.

Contribution

It proposes novel randomized designs specifically tailored for temporal experiments with habituation, achieving minimax optimality without parametric assumptions.

Findings

Designs are proven to be minimax optimal within a broad class of practical designs.

The approach does not rely on parametric modeling of outcomes.

Applicable to experiments with repeated treatments and habituation effects.

Abstract

Randomized experiments are the gold standard for estimating the causal effects of an intervention. In the simplest setting, each experimental unit is randomly assigned to receive treatment or control, and then the outcomes in each treatment arm are compared. In many settings, however, randomized experiments need to be executed over several time periods such that treatment assignment happens at each time period. In such temporal experiments, it has been observed that the effects of an intervention on a given unit may be large when the unit is first exposed to it, but then it often attenuates, or even vanishes, after repeated exposures. This phenomenon is typically due to units' habituation to the intervention, or some other general form of learning, such as when users gradually start to ignore repeated mails sent by a promotional campaign. This paper proposes randomized designs for estimating causal effects in temporal experiments when habituation is present. We show that our designs are minimax optimal in a large class of practical designs. Our analysis is based on the randomization framework of causal inference, and imposes no parametric modeling assumptions on the outcomes.

Equations

$${\text{e}}_t=(0,\dots ,\underset{\text{index}t}{\underbrace{1}},0,\dots ,0).$$

$${\text{w}}_t=(0,\dots ,\underset{\text{index}t}{\underbrace{1}},\dots ,1)=\sum _{s=t}^T{\text{e}}_s.$$

$${\text{Z}}_i={\text{Z}}_i^{\prime }\Rightarrow Y_{it}(\text{Z})=Y_{it}({\text{Z}}^{\prime }),\text{for all}\text{Z},{\text{Z}}^{\prime },\text{and all}t=1,\dots ,T.$$

$$\mathrm{ATE}(t)=\frac{1}{N}\sum _{i=1}^N{Y}_{it}(1)-Y_{it}(0),$$

$$\mathrm{ATE}(t)$$

$$=\frac{1}{N}\sum _{i=1}^N\{Y_{it}(1)-Y_{it}({\text{e}}_t)\}+\frac{1}{N}\sum _{i=1}^N\{Y_{it}({\text{e}}_t)-Y_{it}(0)\}.$$

$${\lambda }_t=\frac{1}{N}\sum _{i=1}^N\{Y_{it}(1)-Y_{it}({\text{e}}_t)\},{\delta }_t=\frac{1}{N}\sum _{i=1}^N\{Y_{it}({\text{e}}_t)-Y_{it}(0)\}.$$

$${\hat{\lambda }}_t=\frac{1}{N_1}\sum _{i=1}^N\text{mathds}1({\text{Z}}_i=1)Y_{it}(1)-\frac{1}{N_{e_t}}\sum _{i=1}^N\text{mathds}1({\text{Z}}_i={\text{e}}_t)Y_{it}({\text{e}}_t),t=2,\dots ,N,$$

$${\hat{\delta }}_t=\frac{1}{N_{e_t}}\sum _{i=1}^N\text{mathds}1({\text{Z}}_i={\text{e}}_t)Y_{it}({\text{e}}_t)-\frac{1}{N_0}\sum _i^N\text{mathds}1({\text{Z}}_i=0)Y_{it}(0),t=2,\dots ,N,$$

$$L(\text{Z},\underline{\text{Y}})=\sum _{t=2}^T({\hat{\lambda }}_t-{\lambda }_t{)}^2+\sum _{t=2}^T({\hat{\delta }}_t-{\delta }_t{)}^2,$$

$$r(\eta ;\underline{\text{Y}})=E_{\eta }\{L(\text{Z},\underline{\text{Y}})\},$$

$${\eta }^{\mathrm{opt}}=\underset{\eta \in \text{mathds}H}{\mathrm{argmin}}\underset{\underline{\text{Y}}\in \underline{\text{mathds}Y}}{\max }\{r(\eta ,\underline{\text{Y}})\}.$$

$$(N_{0}^{\mathrm{opt}},N_{1}^{\mathrm{opt}},\{N_{e_{t}}^{\mathrm{opt}}\}_{t=2}^{T})=\operatorname*{argmin}_{\begin{subarray}{c}N^{\prime}_{0},N^{\prime}_{1},\{N^{\prime}_{e_{t}}\}_{t=2}^{T}\in\mathds{N}^{+}\\ N^{\prime}_{0}+N^{\prime}_{1}+\sum_{t=2}^{T}N^{\prime}_{e_{t}}=N\end{subarray}}\bigg{(}\frac{T-1}{N^{\prime}_{1}}+\frac{T-1}{N^{\prime}_{0}}+2\sum_{t=2}^{T}\frac{1}{N^{\prime}_{e_{t}}}\bigg{)}.$$

$$(\tilde{N}_{0},\tilde{N}_{1},\{\tilde{N}_{e_{t}}\}_{t=2}^{T})=\operatorname*{argmin}_{\begin{subarray}{c}N^{\prime}_{0},N^{\prime}_{1},\{N_{e_{t}}^{\mathrm{opt}}\}_{t=2}^{T}\in\mathds{R}^{+}\\ N^{\prime}_{0}+N^{\prime}_{1}+\sum_{t=2}^{T}N_{e_{t}}=N\end{subarray}}\bigg{(}\frac{T-1}{N^{\prime}_{1}}+\frac{T-1}{N^{\prime}_{0}}+2\sum_{t=2}^{T}\frac{1}{N^{\prime}_{e_{t}}}\bigg{)},$$

$${\tilde{N}}_1={\tilde{N}}_0=\frac{N}{2+\sqrt{2(T-1)}};{\tilde{N}}_{e_t}$$

$$Y_{it}({\text{w}}_{t^{\prime }})=Y_{it}({\text{e}}_{t^{\prime }}),\text{for all},t,t^{\prime }\text{such that}{t}^{\prime }\ge t.$$

$${\hat{\gamma }}_t=\frac{1}{N_{e_t}}\sum _i^N\text{mathds}1({\text{Z}}_i={\text{e}}_t)Y_{it}({\text{e}}_t)-\frac{1}{N_t}\sum _i^N\text{mathds}1(i\in {\mathcal{C}}^{(t)})Y_{it}(0),$$

$$L(\text{Z},\underline{\text{Y}})=\sum _{t=2}^T({\hat{\lambda }}_t-{\lambda }_t{)}^2+\sum _{t=2}^T({\hat{\gamma }}_t-{\delta }_t{)}^2,$$

$$(N_{0}^{\mathrm{opt}},N_{1}^{\mathrm{opt}},\{N_{e_{t}}^{\mathrm{opt}}\}_{t=2}^{T})=\operatorname*{argmin}_{\begin{subarray}{c}N^{\prime}_{0},N^{\prime}_{1},\{N^{\prime}_{e_{t}}\}_{t=2}^{T}\in\mathds{N}\\ N^{\prime}_{0}+N^{\prime}_{1}+\sum_{t=2}^{T}N^{\prime}_{e_{t}}=N\end{subarray}}\bigg{(}\frac{T-1}{N^{\prime}_{1}}+2\sum_{t=2}^{T}\frac{1}{N^{\prime}_{e_{t}}}+\sum_{t=2}^{T}\frac{1}{N^{\prime}_{t}}\bigg{)},$$

$$(N_{0}^{\mathrm{opt}},N_{1}^{\mathrm{opt}},\{N_{e_{t}}^{\mathrm{opt}}\}_{t=2}^{T})=\operatorname*{argmin}_{\begin{subarray}{c}N^{\prime}_{0},N^{\prime}_{1},\{N^{\prime}_{e_{t}}\}_{t=2}^{T}\in\mathds{R}\\ N^{\prime}_{0}+N^{\prime}_{1}+\sum_{t=2}^{T}N^{\prime}_{e_{t}}=N\end{subarray}}\bigg{(}\frac{T-1}{N^{\prime}_{1}}+2\sum_{t=2}^{T}\frac{1}{N^{\prime}_{e_{t}}}+\sum_{t=2}^{T}\frac{1}{N^{\prime}_{e_{t}}}\bigg{)},$$

$$N_{e_t}^{\mathrm{opt}}$$

$$N_1^{\mathrm{opt}}$$

$$N_0^{\mathrm{opt}}$$

$$L(\text{Z},\underline{\text{Y}})=\rho \sum _{t=2}^T({\hat{\lambda }}_t-{\lambda }_t{)}^2+(1-\rho )\sum _{t=2}^T({\hat{\gamma }}_t-{\delta }_t{)}^2,$$

$$(N_{0}^{\mathrm{opt}},N_{1}^{\mathrm{opt}},\{N_{e_{t}}^{\mathrm{opt}}\}_{t=2}^{T})\quad=\operatorname*{argmin}_{\begin{subarray}{c}N^{\prime}_{0},N^{\prime}_{1},\{N^{\prime}_{e_{t}}\}_{t=2}^{T}\in\mathds{N}\\ N^{\prime}_{0}+N^{\prime}_{1}+\sum_{t=2}^{T}N^{\prime}_{e_{t}}=N\end{subarray}}\bigg{(}\rho\frac{T-1}{N^{\prime}_{1}}+\sum_{t=2}^{T}\frac{1}{N^{\prime}_{e_{t}}}+(1-\rho)\sum_{t=2}^{T}\frac{1}{N^{\prime}_{t}}\bigg{)},$$

$$N_{e_t}^{\mathrm{opt}}$$

$$N_1^{\mathrm{opt}}$$

$$N_0^{\mathrm{opt}}$$

$$Y_{it}({\text{e}}_{t^{\prime }})=Y_{it}(0),\forall 0<t^{\prime }\le t-k.$$

$${\hat{\beta }}_t=\frac{1}{N_{e_t}}\sum _i^N\text{mathds}1({\text{Z}}_i={\text{e}}_t)Y_{it}({\text{e}}_t)-\frac{1}{N_{t,k}}\sum _i^N\text{mathds}1(i\in {\mathcal{C}}_k^{(t)})Y_{it}(0),$$

$$(N_{0}^{\mathrm{opt}},N_{1}^{\mathrm{opt}},\{N_{e_{t}}^{\mathrm{opt}}\}_{t=2}^{T})=\operatorname*{argmin}_{\begin{subarray}{c}N^{\prime}_{0},N^{\prime}_{1},\{N_{e_{t}}^{\mathrm{opt}}\}_{t=2}^{T}\in\mathds{N}\\ N^{\prime}_{0}+N^{\prime}_{1}+\sum_{t=2}^{T}N^{\prime}_{e_{t}}=N\end{subarray}}\bigg{(}\frac{T-1}{N^{\prime}_{1}}+2\sum_{t=2}^{T}\frac{1}{N^{\prime}_{e_{t}}}+\sum_{t=2}^{T}\frac{1}{N^{\prime}_{t,k}}\bigg{)}$$

$$L_{\lambda }(\text{Z},\underline{\text{Y}})=\sum _{t=2}^T({\hat{\lambda }}_t-{\lambda }_t{)}^2;L_{\delta }(\text{Z},\underline{\text{Y}})=\sum _{t=2}^T({\hat{\delta }}_t-{\delta }_t{)}^2$$

$$\pi \cdot \text{Z}\equiv [{\text{Z}}_{{\pi }^{-1}(1)},\dots ,{\text{Z}}_{{\pi }^{-1}(N)}],$$

Full text

1. 1 Introduction

2. 2 Setup and notation

3. 2.1 Temporal experiments

4. 2.2 Potential outcomes and estimands

5. 2.2.1 Notation and assumptions.

6. 2.2.2 Estimands.

7. 3 Minimax designs

8. 3.1 Estimators and risk

9. 3.2 Permutation invariance of potential outcomes

10. 3.3 Minimax optimal design

11. 3.4 Minimax optimal design with augmented controls

12. 4 Extensions

13. 4.1 Weighted loss functions

14. 4.2 Recycling units when treatment effect attenuates

15. 5 Simulations

16. 5.1 Treatment allocation

17. 5.2 Maximum risk

18. 6 Concluding remarks

19. 7 Acknowledgments

20. A Proof of results in Section 3.3

21. A.1 Intermediate results and lemmas

22. A.2 Proof of Theorem 1

23. A.3 Proof of Proposition 1

24. B Proof of results in Section 3.4

25. B.1 Proof of Theorem 2

26. B.2 Proof of Proposition 2

27. C Proof of results in Section 4.1

28. C.1 Proof of Theorem 3

29. C.2 Proof of Proposition 3

30. D Proof of results in Section 4.2

31. E Randomization-inference

32. F Expected risk

1 Introduction

In many causal inference problems, it is of interest to understand how treatment effects vary over time. This concerns, for example, the evaluation of the long-term impacts of public policies (Athey et al., 2019; Allcott and Rogers, 2014; Imai et al., 2011; Sjölander et al., 2016; Hainmueller et al., 2014), or applications where treatment effects may carry over from earlier intervention periods, as in clinical trials (Copas et al., 2015; Wellek and Blettner, 2012; Brown Jr, 1980). In this context, the main challenge is that a unit’s current outcome may be affected not just by its current treatment, but also by its past treatments.

For instance, in a recent paper, Allcott and Rogers (2014) studied the impact of mailing a customized energy report to households across the United States on their energy consumption. They repeated this treatment every month on the same households over multiple years, and recorded their monthly energy consumption over the same period. One question at the center of their study was the effect of this repeated treatment, and in particular whether households had habituated to the treatment. Indeed, the authors observed that while households’ energy consumption was reduced upon receiving the first monthly report, the magnitude of the effect shrank dramatically after receiving just a few more reports. Other experiments studying the impact of behavioral interventions for energy conservation have made similar observations (Abrahamse et al., 2005; Hahn and Metcalfe, 2016; Allcott and Mullainathan, 2010). In fact, this phenomenon of habituation after repeated exposure to an intervention has been reported in a variety of contexts, including online advertising (Chatterjee et al., 2003; Hohnhold et al., 2015; Yan et al., 2019), marketing (Wathieu, 2004; Liberali et al., 2011), and ergonomics (Kim and Wogalter, 2009). For instance, in the context of online advertising, Hohnhold et al. (2015) ran a number of experiments to measure “ads blindness”, which is a phenomenon describing the behavior of online users who, when repeatedly exposed to low-quality ads, progressively learn to ignore them. This paper considers the problem of designing optimal experiments for measuring effects of this kind.

Several experimental designs have been applied in practice to quantify habituation under repeated treatments. Hohnhold et al. (2015) proposed two designs ––– the “Post-Period design” and the “Cookie–Cookie-Day Design” ––– for capturing and assessing this phenomenon. The literature on behavioral interventions (e.g., Allcott and Rogers (2014)) also uses a variety of designs with similar ideas. However, the design question is not studied formally in this literature, and optimality properties are not discussed.

In contrast, design optimality is at the heart of the literature on sequential personalized experiments (Lei et al., 2012, for example), where the objective is to adaptively construct a sequence of treatments for each experimental unit in order to maximize an outcome of interest (e.g. blood pressure, quality of life), typically based on covariates and outcomes at previous stages. That setting, however, differs from ours in two fundamental ways. First, the goal of maximizing unit responses is distinct from, and in fact it may conflict with, the goal of estimating temporal causal effects. Second, the literature on adaptive sequential experiments generally requires strong modeling assumptions, whereas our approach makes no modeling assumption on the unit outcomes. Instead, we make use of the randomization framework in causal inference where the potential outcomes are fixed, and randomness comes only from the experimental design.

The goal of this paper is to formalize and develop minimax optimal designs for estimating causal effects in temporal experiments where each unit can be exposed to a different treatment at each time period, and the resulting outcomes are also observed for all units at each time period. We adopt the potential outcomes framework of causal inference (Rubin, 1974; Neyman, 1923), and the randomization-based perspective on inference, extending both to the temporal setting. Methodologically, our setup is therefore closer to that of Bojinov and Shephard (2019) who also consider dynamic potential outcomes and causal effects in temporal experiments. The key difference is that Bojinov and Shephard (2019) consider Fisherian randomization tests for hypotheses on the potential outcomes, whereas we address the design problem of minimax estimation of causal effects. Our study of minimax optimality builds on early work by Wu (1981) and Li et al. (1983) who showed that in the cross-sectional setting, the balanced completely randomized design is minimax optimal. We extend their results to the temporal setting, but find that the minimax optimal design is no longer balanced among all arms — we provide some intuition as to why the symmetry present in the cross-sectional setting is broken in the temporal setting.

The rest of this paper is structured as follows. In Section 2, we describe our framework in detail, and identify two types of temporal causal estimands of interest: habituation effects, as described above, and instantaneous effects, which measure the effect of a single exposure to the treatment at a given time period. We develop the main theory in Section 3, and introduce minimax optimal designs for jointly estimating habituation effect and instantaneous effects. We discuss extensions of our basic design under alternative sets of assumptions in Section 4, and illustrate our results with simulations in Section 5.

2 Setup and notation

2.1 Temporal experiments

We consider a temporal experiment with $N$ units, which is taking place over $T$ discrete periods, indexed by $t=1,\ldots,T$. We assume that $T$ is fixed and known. At each time $t$, a unit $i$ can either receive treatment or control, denoted by $Z_{it}=1$ or $Z_{it}=0$, respectively. Vector $Z_{i,1:t}=(Z_{i1},\ldots,Z_{it})\in\{0,1\}^{t}$ denotes the treatment history of $i$ up to time $t$. The full treatment history for unit $i$ is denoted by $\textbf{Z}_{i}=(Z_{i1},\ldots,Z_{iT})$, where the dependence on $T$ is left implicit; $\textbf{Z}\in\{0,1\}^{N\times T}$ denotes the $N\times T$ matrix of population treatment assignments, whose $i$-th row is equal to $\textbf{Z}_{i}$. A temporal design is thus a distribution on Z.

In our paper, certain assignment vectors play a special role, which requires additional notation. Let $\mathbf{1}=(1,\ldots,1)$ and $\mathbf{0}=(0,\ldots,0)$ be, respectively, the always-treated and always-control assignment vectors of length $T$; as before, dependence on $T$ is left implicit to simplify notation. We call pulse assignment at time $t$, and denote by $\textbf{e}_{t}$, the assignment vector that treats a unit only at time $t$; that is,

[TABLE]

Similarly, the wedge assignment at time $t$, denoted $\textbf{w}_{t}$, treats a unit continuously after $t$:

[TABLE]

The assignment vectors $\mathbf{1}$, $\mathbf{0}$, $\{\textbf{e}_{t}\}_{t=1}^{T}$ and $\{\textbf{w}_{t}\}_{t=1}^{T}$ are the building blocks for the following two classes of designs that will be our main focus of analysis.

Definition 1** (Pulse and wedge designs).**

Let $\mathcal{E}=\{\mathbf{1},\mathbf{0},\textbf{e}_{1},\ldots,\textbf{e}_{T}\}$. Denote by $\mathcal{Z}(\mathcal{E})=\{\textbf{Z}\in\{0,1\}^{N\times T}:\textbf{Z}_{i}\in\mathcal{E},~{}\forall i=1,\ldots,N\}$ the set of all assignment matrices for which every unit’s assignment is in $\mathcal{E}$. Similarly, let $\mathcal{E}^{\ast}=\{\mathbf{1},\mathbf{0},\textbf{w}_{1},\ldots,\textbf{w}_{T}\}$ and denote by $\mathcal{Z}(\mathcal{E}^{\ast})=\{\textbf{Z}\in\{0,1\}^{N\times T}:\textbf{Z}_{i}\in\mathcal{E}^{\ast},~{}\forall i=1,\ldots,N\}$ the set of all assignment matrices for which every unit assignment is in $\mathcal{E}^{\ast}$. A pulse design is a probability distribution, $\eta(\textbf{Z})$, with support on $\mathcal{Z}(\mathcal{E})$. A wedge design is a probability distribution, $\eta_{\mathrm{w}}(\textbf{Z})$, with support on $\mathcal{Z}(\mathcal{E}^{\ast})$. The sets of all possible pulse designs and wedge designs are denoted by $\mathds{H}$ and $\mathds{H}^{\ast}$, respectively.

Pulse and wedge designs are the building blocks of our causal estimands (see Section 2.2.2). They are natural classes of designs to consider, as they arise in many applications other than the ones we have considered so far; for instance, wedge designs are popular in clinical trials (Brown and Lilford, 2006; Prost et al., 2015; Hargreaves et al., 2015). In online advertising, the Cookie–Cookie-Day design (Hohnhold et al., 2015) is an example of a wedge design.

In the following section, we focus on pulse designs because they are conceptually simpler, but our key results hold unchanged for wedge designs as well (see Remark 4).

2.2 Potential outcomes and estimands

To define our causal estimands, we adapt the classical potential outcomes framework of causal inference (Neyman, 1923; Rubin, 1974) to the temporal setting, building on recent related work (Ji et al., 2017; Bojinov and Shephard, 2019). In this framework, the potential outcomes are fixed, and randomness comes exclusively from the random assignment Z.

2.2.1   Notation and assumptions.

Let $Y_{it}(\textbf{Z})$ denote the scalar potential outcome of unit $i$ at time period $t$ under population assignment Z. We also denote by $\textbf{Y}_{i}(\textbf{Z})=(Y_{i1}(\textbf{Z}),\ldots,Y_{iT}(\textbf{Z}))$ the vector of outcomes over time under Z for unit $i$, and by $\textbf{Y}(\textbf{Z})=[\textbf{Y}_{1}(\textbf{Z}),\ldots,\textbf{Y}_{N}(\textbf{Z})]^{\top}$ the $N\times T$ matrix whose $i$-th row is equal to $\textbf{Y}_{i}(\textbf{Z})$. We make the following two assumptions.

Assumption 1** (No interference).**

The outcome of unit $i$ at time $t$ depends only on it’s own sequence of assignments. That is, for every unit $i$,

[TABLE]

Assumption 2** (Non-anticipating outcome).**

The outcome of unit $i$ at time $t$ depends only on the treatment history up to time $t$.

Assumption 1 extends to the temporal setting the classical “no interference” assumption (Cox, 1958), which is standard in causal inference and econometrics (Imbens and Rubin, 2015). Assumption 2 does not allow the potential outcomes at $t$ to depend on future treatments after $t$ (Bojinov and Shephard, 2019), and is standard in classical econometric studies  (Heckman and Vytlacil, 2005; Heckman et al., 2016; Robins, 1997; Toh and Hernán, 2008). This assumption could fail, for example, in settings where experimental units are made aware of (and can therefore anticipate) the future sequence of treatments. In this setting, the units’ outcomes may reflect not only the effect of past treatments, but also the units’ expectations of future treatments, complicating the analysis.

2.2.2   Estimands.

In the potential outcomes framework, the estimands of interest are defined as contrasts of potential outcomes (Rubin, 1974). In our setting, a simple causal estimand is the average treatment effect at time $t$, namely,

[TABLE]

which contrasts the potential outcomes at time $t$ between the always-treated and always-control population assignments. This estimand entangles the effects of past treatments with the effect of the current treatment, that is, it is a combination of habituation effects and instantaneous effects. Formally, we can write:

[TABLE]

This decomposition leads to the following definitions.

Definition 2**.**

For a given time period $t$, the habituation effect, $\lambda_{t}$, and the instantaneous effect, $\delta_{t}$, are defined as:

[TABLE]

As defined, the habituation effect, $\lambda_{t}$, contrasts the potential outcomes under the always-treated assignment and the pulse assignment at time period $t$. These two assignments have the same treatment at time $t$, so $\lambda_{t}$ intuitively captures the treatment effect that can be attributed only to a cumulative effect from treatment history and not to treatment application at time $t$. With this definition, we aim to capture the habituation effects studied in the literature (Hohnhold et al., 2015; Allcott and Rogers, 2014). The instantaneous effect, $\delta_{t}$, contrasts the potential outcomes under the pulse assignment and the always-control assignment. These two assignments have the same treatment history up to time $t-1$ but differ in their treatment at time $t$. Thus, $\delta_{t}$ captures the effect of the treatment that can be attributed only to its application at time $t$, and not to any carryover effects from past treatments. For the rest of this paper, our goal will be to propose minimax optimal designs for estimating the habituation and instantaneous effects jointly.

Remark 1**.**

The definitions of the habituation  and instantaneous effects involve only the potential outcomes $Y_{it}(\mathbf{1})$, $Y_{it}(\mathbf{0})$ and $Y_{it}(\textbf{e}_{t})$; these potential outcomes derive from the assignments $\mathbf{1}$, $\mathbf{0}$ and $\{\textbf{e}_{t}\}$, which form the backbone of (and therefore justify the use of) the pulse design.

Remark 2**.**

In the context of online advertisement, interest in the habituation effect has been motivated by the goal of estimating the long-term causal effects of an intervention based on a short-term experiment (Hohnhold et al., 2015). We believe that even if the ultimate objective is to estimate the long-term average treatment effect $\mathrm{ATE}(T^{\ast})$ for $T^{\ast}>T$, one should still focus on the estimands $\{\lambda_{t}\}_{t=1}^{T}$ and $\{\delta_{t}\}_{t=1}^{T}$. Indeed, we argue that extrapolation of the $\mathrm{ATE}$ should be driven mostly by an extrapolation of the habituation effects $\{\lambda_{t}\}_{t=1}^{T}$ which, in the context of online advertisement, can be supported by behavioral game theory (Chatterjee et al., 2003; Toulis and Parkes, 2016). In contrast, the instantaneous effects $\{\delta_{t}\}_{t=1}^{T}$ are largely conjectural, and it is difficult to justify their extrapolation. In this context, our recommendation is to first estimate both the habituation and instantaneous effects, then to verify that the instantaneous effects are stationary, and, finally, to extrapolate the habituation effects. We leave this for future work.

3 Minimax designs

3.1 Estimators and risk

To state our minimax results, we need to define estimators for our estimands, $\delta_{t},\lambda_{t}$. To estimate $\lambda_{t}$ we define the following plug-in estimator,

[TABLE]

where $N_{1}=\sum_{i=1}^{N}\mathds{1}(\textbf{Z}_{i}=\mathbf{1})$ is the number of always-treated units, and $N_{e_{t}}=\sum_{i=1}^{N}\mathds{1}(\textbf{Z}_{i}=\textbf{e}_{t})$ is the number of units assigned to a pulse at time $t$. Variables $N_{1},N_{e_{t}}$ are random because Z is random in the experiment. Similarly, define the plug-in estimator of $\delta_{t}$ as follows,

[TABLE]

where $N_{\mathbf{0}}=\sum_{i=1}^{N}\mathds{1}(\textbf{Z}_{i}=\mathbf{0})$ is the number of always-control units. These plug-in estimators are simple and have well-studied sampling properties under randomization (Imbens and Rubin, 2015). In addition, their symmetry makes them amenable to minimax analysis.

The risk of these estimators is a function of the design and potential outcomes. Let $\textbf{Y}(\textbf{z})=[\textbf{Y}_{1}(\textbf{z})~{}\ldots~{}\textbf{Y}_{N}(\textbf{z})]^{\top}$ denote the $N\times T$ matrix of potential outcomes under population assignment z, whose $i$-th row is equal to $\textbf{Y}_{i}(\textbf{z})$. The full schedule of potential outcomes, denoted by $\underline{\textbf{Y}}=[\textbf{Y}(\mathbf{0}),\textbf{Y}(1),\textbf{Y}(\textbf{e}_{2}),\ldots,\textbf{Y}(\textbf{e}_{T})]$, therefore contains all the information needed for causal inference, since the causal estimands, $\lambda_{t}$ and $\delta_{t}$, are deterministic functions of $\underline{\textbf{Y}}$. For a random assignment $\textbf{Z}\in\mathcal{Z}(\mathcal{E})$ and a schedule of potential outcomes $\underline{\textbf{Y}}$, we consider the squared loss function,

[TABLE]

which is a function only of Z and $\underline{\textbf{Y}}$, since $\widehat{\lambda}_{t}$ and $\widehat{\delta}_{t}$ are deterministic functions of Z and $\underline{\textbf{Y}}$, while $\lambda_{t}$ and $\delta_{t}$ are functions of $\underline{\textbf{Y}}$ only. The risk of a pulse design, $\eta\in\mathds{H}$, is then defined as

[TABLE]

where the expectation is with respect to the randomization distribution in the design, $\eta(\textbf{Z})$.

As mentioned in Section 2.2, the schedule of potential outcomes, $\underline{\textbf{Y}}$, is considered fixed, all randomness coming from Z. A minimax pulse design, $\eta^{\mathrm{opt}}$, minimizes the maximum risk over the support of potential outcome schedules, denoted by $\underline{\mathds{Y}}$, over the entire class of pulse designs $\mathds{H}$, that is, we define

[TABLE]

The task of obtaining a minimax design can therefore be seen as a game between the statistician who chooses a pulse design $\eta\in\mathds{H}$, and nature who chooses the worst-case schedule of potential outcomes from $\underline{\mathds{Y}}$. To make progress, we impose an invariance property on $\underline{\mathds{Y}}$, which we describe in the following section.

3.2 Permutation invariance of potential outcomes

In the cross-sectional setting, Wu (1981) considered minimax designs over permutation-invariant sets of model parameters to reflect the experimenter’s ignorance at the design stage. We adapt this idea to our setting by introducing a similar notion of permutation invariance for $\underline{\mathds{Y}}$.  Specifically, for some set $\mathcal{Y}\subset\mathbb{R}^{N}$, let $\mathbb{Y}(\mathcal{Y})=\{[\textbf{y}_{1}~{}\ldots~{}\textbf{y}_{T}]:\textbf{y}_{t}\in\mathcal{Y},~{}t=1,\ldots,T\}$ be the set of all $N\times T$ matrices whose columns are in $\mathcal{Y}$, and let $\underline{\mathds{Y}}(\mathcal{Y})$ be the set of all potential outcomes schedules whose matrices are all elements of $\mathbb{Y}(\mathcal{Y})$. Thus, if $\underline{\textbf{Y}}=[\textbf{Y}(\mathbf{0}),\textbf{Y}(1),\textbf{Y}(\textbf{e}_{2}),\ldots,\textbf{Y}(\textbf{e}_{T})]\in\underline{\mathds{Y}}(\mathcal{Y})$ is one such schedule of potential outcomes, then $\textbf{Y}(\textbf{z})\in\mathbb{Y}(\mathcal{Y})$ for all $\textbf{z}\in\mathcal{E}$.

Definition 3** (Permutation-invariant schedule).**

Let $\mathsf{S}_{N}$ be the symmetric group on $N$ elements. A set $\underline{\mathds{Y}}$ of potential outcomes schedules is called permutation-invariant if $\underline{\mathds{Y}}=\underline{\mathds{Y}}(\mathcal{Y})$ for some $\mathcal{Y}\subset\mathbb{R}^{N}$, such that $\mathsf{S}_{N}\cdot\mathcal{Y}=\mathcal{Y}$, where $\mathsf{S}_{N}\cdot\mathcal{Y}$ is the set in which every element of $\mathcal{Y}$ has been permuted with every element of $\mathsf{S}_{N}$.

The permutation invariance property in Definition 3 captures a form of symmetry on the units’ outcomes. For example, it implies that for any $\textbf{z}\in\mathcal{E}$ and any $t=1,\ldots,T$, if it is possible that $\textbf{y}_{t}(\textbf{z})=\textbf{y}$, where $\textbf{y}_{t}(\textbf{z})=(Y_{1t}(\textbf{z}),\ldots,Y_{Nt}(\textbf{z}))$, it should also be possible that $\textbf{y}_{t}(\textbf{z})=\pi\cdot\textbf{y}$, for any permutation $\pi\in\mathsf{S}_{N}$. This property is not a probabilistic statement about the likelihood of y or $\pi\cdot\textbf{y}$, but a statement about the support of the potential outcomes, which is a weaker assumption; see also (Wu, 1981) for an interpretation of permutation invariance in terms of robustness.

3.3 Minimax optimal design

We can now state our first minimax theorem for pulse designs. The resulting design is the solution to an integer optimization problem, which is generally hard to solve. We therefore discuss a continuous relaxation in Proposition 1.

Theorem 1** (Minimax pulse design).**

Let $\underline{\mathds{Y}}$ be a bounded, permutation-invariant set of potential outcome schedules. The minimax optimal pulse design, $\eta^{\mathrm{opt}}$, in Equation (8) is the completely randomized design that assigns $N_{1}^{\mathrm{opt}}$, $N_{0}^{\mathrm{opt}}$ and $\{N_{e_{t}}^{\mathrm{opt}}\}_{t=2}^{T}$ units to the assignments $\mathbf{1}$, $\mathbf{0}$, and $\{\textbf{e}_{t}\}_{t=2}^{T}$, respectively, where:

[TABLE]

Two aspects of this result deserve special mention. First, the minimax design in Theorem 1 is a completely randomized design. This result agrees with the results of Wu (1981) and Li et al. (1983) who proved that complete randomization is minimax optimal in the static, cross-sectional setting. Second, in contrast to the aforementioned works, the minimax design in Theorem 1 is not balanced as it does not assign the same number of units to each of the $T+1$ treatment arms in $\mathcal{E}$. This is made clearer by the following result.

Proposition 1**.**

The relaxation of the integer optimization problem of Equation (9),

[TABLE]

has the following solutions:

[TABLE]

The solution of the relaxed problem exhibits a partial asymmetry with three notable features. First, $\tilde{N}_{1}=\tilde{N}_{0}$ while $\tilde{N}_{e_{t}}=\tilde{N}_{e_{t}^{\prime}}$ for all $1\leq t,t^{\prime}\leq T$, and so there is symmetry between always-control and always-treated units, but not across all units. Second, $\tilde{N}_{\textbf{e}_{t}}/\tilde{N}_{\mathbf{1}}\to 0$ as the time horizon $T$ grows, and so the number of units assigned to any pulse assignment is asymptotically negligible compared to the always-treated (or always-control) assignments. Third, $\tilde{N}_{\mathbf{1}}/\sum_{t=2}^{T}\tilde{N}_{\textbf{e}_{t}}\to 0$, and so the pulse assignments taken together dominate the other treatment arms. To get intuition, let us consider the problem with $T=30$ and $N=10000$. In this case, the minimax design assigns $\tilde{N}_{1}=\tilde{N}_{0}\approx 1040$, and $\tilde{N}_{e_{t}}\approx 273$ for $t=2,\ldots,30$. In contrast, a balanced randomized design would assign $322$ units to each treatment arm. Such difference between our design and standard balanced designs mainly stem from the definition of our loss function in Equation (6), which takes into account arbitrary-sized effects from past treatment history.

Remark 3**.**

Since the optimal design obtained in Theorem 1 is completely randomized, randomization-based analysis of the experiment is possible using standard Neymanian theory (Imbens and Rubin, 2015, Chapter 6). See proof in Appendix for details.

Remark 4** (Wedge designs).**

As mentioned earlier, the results in this section (as well as in Section 3.4) are stated in terms of pulse designs, but they can be extended to wedge designs since, by Assumption 2,

[TABLE]

That is, the potential outcomes of a unit under assignments $\textbf{w}_{t^{\prime}}$ and $\textbf{e}_{t^{\prime}}$ are identical for all periods prior to and including $t^{\prime}$. In particular, the estimands, $\lambda_{t}$ and $\delta_{t}$, can be written in terms of wedge assignments instead of pulse assignments by substituting $\textbf{w}_{t}$ for $\textbf{e}_{t}$ in Equation (2), and the rest of the analysis remains unchanged.

3.4 Minimax optimal design with augmented controls

The design of Theorem 1 was obtained using plug-in estimators for $\delta_{t}$ and $\lambda_{t}$. Here, we discuss the minimax design problem using a better estimator for $\delta_{t}$. The key idea relies on Assumption 2, which implies that $Y_{it}(\textbf{e}_{t^{\prime}})=Y_{it}(\mathbf{0})$, for all $t<t^{\prime}$. In other words, under Assumption 2, at all times prior to $t^{\prime}$ a pulse assignment $\textbf{e}_{t^{\prime}}$ is indistinguishable from an always-control assignment in the sense that a unit assigned to $\textbf{e}_{t^{\prime}}$ behaves as if it had been assigned to $\mathbf{0}$, for all $t<t^{\prime}$. We can therefore use outcomes from units assigned to pulses for estimation of unknown control outcomes.

The new estimator that replaces the plug-in estimator, $\hat{\delta}_{t}$, is defined as follows:

[TABLE]

where $N_{t}=|\mathcal{C}^{(t)}|$, and $\mathcal{C}^{(t)}=\{i:\textbf{Z}_{i}=\mathbf{0},~{}\text{or}~{}\textbf{Z}_{i}=e_{t^{\prime}},~{}t^{\prime}>t\}$. Here, $\mathcal{C}^{(t)}$ is the new set of “augmented control” units at $t$, i.e., the set comprised either of always-control units, or units assigned to pulse at a time $t^{\prime}>t$. The new loss function is now defined as

[TABLE]

which only differs from Equation (7) in using the new estimator, $\hat{\gamma}_{t}$. The updated minimax result is stated in the following theorem.

Theorem 2**.**

Let $\underline{\mathds{Y}}$ be a bounded, permutation-invariant set of potential outcome schedules. The minimax optimal pulse design, $\eta^{\mathrm{opt}}$, in Equation (8) using the new instantaneous effect estimator $\hat{\gamma}_{t}$ in Equation (11), is the completely randomized design that assigns $N_{1}^{\mathrm{opt}}$, $N_{0}^{\mathrm{opt}}$ and $\{N_{e_{t}}^{\mathrm{opt}}\}_{t=2}^{T}$ units to the assignments $\mathbf{1}$, $\mathbf{0}$, and $\{\textbf{e}_{t}\}_{t=2}^{T}$, respectively, where:

[TABLE]

and for each $t$, $N^{\prime}_{t}=|\mathcal{C}^{(t)}|=N_{0}+\sum_{t^{\prime}=2}^{T}\mathds{1}(t^{\prime}>t)N_{e_{t^{\prime}}}$.

The main practical difference between Theorem 2 and Theorem 1 is that the optimization problem of Equation (13) is replacing that of Equation (10). The following proposition derives the solutions to a continuous relaxation of the problem.

Proposition 2**.**

The following integer relaxation of the optimization problem in Equation (13),

[TABLE]

has analytical solutions:

[TABLE]

where $\{c_{t}\}_{t=2}^{T}$ are defined recursively by $c_{T}=1$ and c_{t}=\bigg{[}\frac{1}{c_{t+1}^{2}}+\frac{1}{(1+\sqrt{2}\sum_{t^{\prime}>t}c_{t^{\prime}})^{2}}\bigg{]}^{-1/2}, for all $t=2,\ldots,T-1$.

The solution described by Proposition 2 offers a sharp contrast to the solution described by Proposition 1. Specifically, the solution of Proposition 2 does not exhibit the same form of partial symmetry as in Proposition 1 because the new estimator, $\hat{\gamma}_{t}$, uses outcomes from pulse treatment as information about control potential outcomes. This reflects the fundamental asymmetry of the loss function in Equation (12) in how it uses always-treated and always-control units. The effect will be illustrated more clearly in the simulations of Section 5, which will give more insight into this minimax design by comparing it to the minimax design of Section 3.3, and to the standard, balanced completely randomized design.

4 Extensions

4.1 Weighted loss functions

The loss functions considered in Equation (6) and Equation (12) put the same weight on the instantaneous effects and the habituation effects. This implicitly assumes that both types of effects are of equal interest, which may not be true in practice. More flexible loss functions could assign different weights to instantaneous and habituation effects. In this section, we focus on extending the results of Section 3.4; analogous results for Section 3.3 can be derived.

Specifically, for some $\rho\in[0,1]$ consider the weighted loss function,

[TABLE]

which generalizes Equation (12). Thus, parameter $\rho$ controls the relative importance in estimating $\lambda_{t}$ or $\delta_{t}$. The original loss function in Equation (6) is a special case (up to a multiplicative constant) with $\rho=1/2$. The following theorem extends Theorem 2 to this new loss function, and derives the minimax optimal design as a function of $\rho$.

Theorem 3**.**

Under the weighted loss function of Equation (14), the minimax optimal design, $\eta^{\mathrm{opt}}$, is still completely randomized, but with $N_{1}^{\mathrm{opt}}$, $N_{0}^{\mathrm{opt}}$ and $\{N_{e_{t}}^{\mathrm{opt}}\}_{t=2}^{T}$ solving:

[TABLE]

where for each $t$, $N^{\prime}_{t}=|\mathcal{C}^{(t)}|=N_{0}+\sum_{t^{\prime}=2}^{T}\mathds{1}(t^{\prime}>t)N_{e_{t^{\prime}}}$.

In words, the minimax optimal design under a weighted loss function is still completely randomized: what changes is the number of units assigned to each arm, depending on the parameter $\rho$. The following relaxation of the optimization problem in Equation (15) provides more intuition.

Proposition 3**.**

The continuous relaxation of the optimization problem of Equation (15) has solutions, for $\rho<1$:

[TABLE]

where $\ell=(1-\rho)^{-1/2}$, $c_{T}=1$ and c_{t}=\bigg{[}\frac{1}{c^{2}_{t+1}}+\frac{1}{(1+\ell\sum_{t^{\prime}>t}c_{t^{\prime}})^{2}}\bigg{]}^{-1/2}, for all $t=2,\ldots,T-1$. The case when $\rho=1$ is obtained by symmetry with $\rho=0$; see Appendix.

It is straightforward to see that when $\rho=1/2$ we recover the results of Proposition 2. Additional intuition can be obtained by examining boundary values of $\rho$. Consider, for example, the case when $\rho=1$, such that the loss function only involves habituation effects. Then, $N_{0}^{\mathrm{opt}}=0$. This is reasonable: if we are only interested in estimating habituation effects, then always-control units are not needed. Similarly, when the loss function only involves instantaneous effects ($\rho=0$), the minimax optimal design assigns no units to the always-treated arm ($N_{1}=0$).

4.2 Recycling units when treatment effect attenuates

A fundamental premise of our approach so far is that we allow the effects of a treatment to persist indefinitely. In some cases, however, it may be reasonable to assume that the effect of the treatment wears off if a unit is untreated for a certain length of time, after being treated. We state this assumption formally, and then derive the resulting minimax design.

Assumption 3** ($k$-order carryovers).**

For all $i=1,\ldots,N$ and all $t=1,\ldots,T$,

[TABLE]

Assumption 3 implies that after $k$ periods following a pulse treatment, $\textbf{e}_{t^{\prime}}$, the effects of the pulse assignment are indistinguishable from those of an always-control treatment. Thus, a unit assigned to a pulse $\textbf{e}_{t^{\prime}}$ behaves as if it had been assigned to $\mathbf{0}$, for all time periods after (and including) $t^{\prime}+k$. Taken together, Assumption 2 and Assumption 3 suggest a new estimator of the instantaneous effect that generalizes the “recycling estimator” of Section 3.4.

In particular, for some time period $t$, and $k\in\{1,\ldots,T\}$ with $t-k>0$, let $\mathcal{C}_{k}^{(t)}(\underline{\textbf{Z}})=\{i:\textbf{Z}_{i}=\mathbf{0}\,\text{ or }\textbf{Z}_{i}=e_{t^{\prime}},\,\,\text{ with }\,t^{\prime}\leq t-k\,\text{ or }t^{\prime}>t\}$ be the new set of “augmented controls”, comprised of units assigned either to always-control, to pulses before $t-k$, or to pulses after $t$. We replace the estimator of the instantaneous effect, $\hat{\delta}_{t}$, by the following estimator:

[TABLE]

where $N_{t,k}=|\mathcal{C}_{k}^{(t)}|$. This new estimator is “recycling” units that are assigned to pulses in order to estimate control outcomes (term $\mathcal{C}^{(t)}_{k}$ in Equation (16)). The loss and risk functions are as in Section 3, but with $\hat{\beta}_{t}$ instead of $\hat{\delta}_{t}$ (or $\hat{\gamma}_{t}$). We can now state the minimax result under the new estimator of the instantaneous effect.

Theorem 4**.**

Let $\underline{\mathds{Y}}$ be a bounded, permutation-invariant set of potential outcome schedules. The minimax optimal pulse design, $\eta^{\mathrm{opt}}$, in Equation (8) using the new instaneous effect estimator $\hat{\beta}_{t}$ in Equation (16) is the completely randomized design that assigns $N_{1}^{\mathrm{opt}}$, $N_{0}^{\mathrm{opt}}$ and $\{N_{e_{t}}^{\mathrm{opt}}\}_{t=2}^{T}$ units to the assigmnents $\mathbf{1}$, $\mathbf{0}$, and $\{\textbf{e}_{t}\}_{t=2}^{T}$, respectively, where:

[TABLE]

As before the optimal design is completely randomized but the number of units assigned to each treatment arm differs from previous designs. The integer optimization problem it involves (as well as its relaxation) is difficult to solve analytically. We plan to address this problem in future work.

Remark 5**.**

The results of this section are specific to pulse designs in contrast to the results of Sections 3.3, 3.4 and 4.1, which also apply to wedge designs. Indeed, the fundamental idea of “recycling” units is that if we wait long enough after a pulse, the units assigned to the pulse behave like control units. This does not apply when wedge designs are used, since units remain treated after the pulse.

5 Simulations

This section illustrates visually two aspects of our theory. In Section 5.1, we compute the number of units allocated to each treatment arms for three designs: the balanced completely randomized design (BCRD) that assigns the same number of units to all $T+1$ treatment arms, the minimax design of Section 3.3 and the augmented minimax design of Section 3.4. In Section 5.2 we quantify the reduction in maximum risk from using our minimax designs, compared to the BCRD. This paper focuses exclusively on designs that minimize the maximum risk: our theory says nothing about the expected risk. One cannot reasonably expect minimax optimal designs to also be optimal for minimizing the expected risk — this is the price to pay for generality. Nevertheless, in Appendix, we compare the expected risk of our minimax design to that of the BCRD under two simple models, and show that our designs improve the expected risk slightly.

5.1 Treatment allocation

We start by illustrating visually the optimal allocations obtained analytically in Proposition 1 and Proposition 2. Figure 1 shows the optimal allocation for the BCRD, the minimax design and the augmented minimax design for $N=10000$ and for $T=5,10,15$.

As expected, the BCRD produces a fully symmetric allocation, shown as a horizontal blue line. The standard minimax design of Section 3.3 produces an allocation that is only partially symmetric between two groups: $N_{1}=N_{0}$ and $N_{e_{t}}=N_{e_{t}^{\prime}}$ for all $2\leq t,t^{\prime}\leq T$; see Proposition 1 and the subsequent discussion for details on such symmetry. We also confirm visually that the minimax optimal design allocates more units than the BCRD to the always-treated arm $(\textbf{z}=\mathbf{1})$ and always-control arm $(\textbf{z}=\mathbf{0})$, but less in the pulses arms $(\textbf{z}=\textbf{e}_{t})$. On the other hand, the minimax design with augmented controls does not exhibit a symmetry. It allocates very few units to the always-control arms: this is expected since in this setting, some pulse units can be used as controls at each time $t$. We also see that the number of units assigned to the pulse assignments $\{\textbf{e}_{t}\}_{t=2}^{T}$ increases for larger values of $t$. This again is consistent with our intuition: as $t$ increases, the number of time periods for which a pulse can be used as control increases.

5.2 Maximum risk

The risk under the augmented minimax design should be lower than the risk under the minimax design, which in turn should be lower than the risk under BCRD. In this section, we illustrate the magnitude of that reduction. Figure 2 plots the maximum risk for the minimax and augmented minimax designs relative to the maximum risk of BCRD, for $N=1000$ and $T=10,20,30,40,50$.

We consider two settings, as shown in Figure 2. In the left panel, the estimator $\hat{\gamma}$ is used for the risk under the augmented minimax but not for the risk under the other designs; in the right panel, the estimator $\hat{\gamma}$ is used with all three designs. In both settings, the maximum risk is always lower for the augmented minimax design as predicted by theory. Compared to BCRD, the augmented minimax design reduces the risk by up to $20\%$ compared to when $\hat{\lambda}$ is used for all designs. Given the trend further reduction would be expected for longer time horizons.

6 Concluding remarks

In this paper, we have constructed minimax optimal designs for estimating the habituation  and instantaneous effects in temporal experiments. Our construction uses the potential outcomes framework of causal inference, and is nonparametric with mild assumptions on the support of potential outcomes.

There are several open questions for future work. First, from a technical perspective, it would be interesting to obtain analytical solutions for the recycling designs in Theorem 4. These should outperform the augmented minimax design because they wouldn’t throw away useful data. Second, in future work we would like to connect more formally our estimands ($\lambda_{t},\delta_{t}$ in Definition 2) to estimands of long-term effects in user behavior experiments, which are popular in digital experimentation (Hohnhold et al., 2015; Yan et al., 2019; Kohavi et al., 2009). Third, we are looking at extensions of our approach, allowing covariate information to be incorporated in the design. Finally, our designs minimize the maximum risk, but they may be far from optimal with respect to the expected risk. In future work, we plan on exploring optimal designs for complex models of habituation.

7 Acknowledgments

We would like thank organizers and participants at the Google Market Workshop, and at the LinkedIn Seminar for useful comments and feedback.

Appendix A Proof of results in Section 3.3

A.1 Intermediate results and lemmas

The proof of Theorem 1 has a number of intermediate steps, which we will state as lemmas.

First, we define:

[TABLE]

so that we have $L(\textbf{Z},\underline{\textbf{Y}})=L_{\lambda}(\textbf{Z},\underline{\textbf{Y}})+L_{\delta}(\textbf{Z},\underline{\textbf{Y}})$. Next, we need to define the permutation actions on vectors and matrices.

Definition 4**.**

Let $\mathsf{S}_{N}$ be the symmetric group on $N$ elements, and $\pi\in\mathsf{S}_{N}$. If $\mathbf{v}$ is a vector of length $N$, then the action of $\pi$ on $\mathbf{v}$, denoted $\pi\cdot\mathbf{v}$, is the vector obtained by permuting the indices of $\mathbf{v}$ according to $\pi$. Similarly, for any population assignment $\textbf{Z}=[\textbf{Z}_{1},\ldots,\textbf{Z}_{N}]\in\mathcal{Z}(\mathcal{E})\subset\{0,1\}^{N\times T}$,

[TABLE]

where $\pi^{-1}(i)$ is the element $j$ that is mapped to $i$ through $\pi$.

Definition 5**.**

For a potential outcomes schedule, $\underline{\textbf{Y}}=[\textbf{Y}(\mathbf{0}),\textbf{Y}(\mathbf{1}),\textbf{Y}(\textbf{e}_{2}),\ldots,\textbf{Y}(\textbf{e}_{T})]$, we define

[TABLE]

where, as above, we have $\pi\cdot\textbf{Y}(\textbf{z})\equiv(\textbf{Y}_{\pi^{-1}(1)}(\textbf{z}),\ldots,\textbf{Y}_{\pi^{-1}(N)}(\textbf{z}))$, for every assignment $\textbf{z}\in\mathcal{E}$.

We can now state and prove a sequence of lemmas. Throughout, we denote $\mathds{I}=\{1,\ldots,N\}$. It is immediate to see that if $\pi\in\mathsf{S}_{N}$, then $\pi\cdot\mathds{I}\equiv\{\pi(i),i\in\mathds{I}\}=\mathds{I}$ and thus $\pi^{-1}\cdot\mathds{I}=\mathds{I}$. Our first lemma is to show that the loss function, $L$, is permutation-invariant in its arguments.

Lemma 1**.**

For any $\pi\in\mathsf{S}_{N}$, and any assignment matrix Z and schedule $\underline{\textbf{Y}}$,

[TABLE]

Proof.

We show that $L_{\delta}(\pi\cdot\textbf{Z},\pi\cdot\underline{\textbf{Y}})=L_{\delta}(\textbf{Z},\underline{\textbf{Y}})$. The proof for $L_{\lambda}$ is identical, and so the proof for $L$ follows immediately.

Recall that $\delta_{t}(\underline{\textbf{Y}})=N^{-1}\sum_{i}Y_{it}(\textbf{e}_{t})-N^{-1}\sum_{i}Y_{it}(\mathbf{0})$. We have:

[TABLE]

We now turn to the estimator $\hat{\delta}_{t}$,

[TABLE]

With a similar argument as before:

[TABLE]

The exact same reasoning leads to $\hat{\delta}_{t}^{(\mathbf{0})}(\pi\cdot\textbf{Z},\pi\cdot\underline{\textbf{Y}})=\hat{\delta}_{t}^{(\mathbf{0})}(\textbf{Z},\underline{\textbf{Y}})$, and so:

[TABLE]

Putting together Equation (18) and Equation (19) we have:

[TABLE]

∎

Lemma 2**.**

For a pulse design $\eta\in\mathds{H}$ and $\pi\in\mathsf{S}_{N}$, let $\eta_{\pi}$ be the design such that $\eta_{\pi}(\textbf{Z})=\eta(\pi\cdot\textbf{Z})$, and let $\tilde{\eta}=(N!)^{-1}\sum_{\pi\in\mathsf{S}_{N}}\eta_{\pi}$. Then, if $\underline{\mathds{Y}}(\mathcal{Y})$ is permutation-invariant, we have:

[TABLE]

Proof.

We have:

[TABLE]

It follows that

[TABLE]

where the last equality follows from the fact that the support $\mathcal{Z}(\mathcal{E})$ is permutation invariant. Now we use the fact that $\underline{\mathds{Y}}(\mathcal{Y})$ is permutation invariant:

[TABLE]

Putting everything together, we obtain:

[TABLE]

∎

Next, we prove a representation lemma.

Lemma 3**.**

Let $\delta_{\textbf{Z}}$ be the design that assigns mass 1 at the assignment Z. Let $\eta\in\mathds{H}$. Then:

[TABLE]

where $\tilde{\delta}_{\textbf{Z}}=(N!)^{-1}\sum_{\pi\in\mathsf{S}_{N}}\xi_{\pi,\textbf{Z}}$ and $\xi_{\pi,\textbf{Z}}(\textbf{Z}^{\prime})=\delta_{\textbf{Z}}(\pi\cdot\textbf{Z}^{\prime})$.

Proof.

From $\pi\cdot\mathcal{Z}(\mathcal{E})=\mathcal{Z}(\mathcal{E})$ it follows, for all $\pi\in\mathsf{S}_{N}$, that

[TABLE]

By definition, $\delta_{\pi^{-1}\cdot\textbf{Z}}=\xi_{\pi,\textbf{Z}}$ since both function out mass 1 at the treatment $\pi^{-1}\cdot\textbf{Z}$. Then, from its definition in Lemma 2:

[TABLE]

∎

Lemma 4**.**

Let $\eta\in\mathds{H}$ and $\underline{\mathds{Y}}(\mathcal{Y})$ be a permutation-invariant schedule of potential outcomes, where $\mathcal{Y}$ is bounded. Then,

[TABLE]

where

[TABLE]

Proof.

We have:

[TABLE]

where $\tilde{\delta}_{\textbf{Z}}$ is the CRD that assigns $N_{0}(\textbf{Z})$ units to $\mathbf{0}$, $N_{1}(\textbf{Z})$ units to $\mathbf{1}$, and $N_{e_{t}}(\textbf{Z})$ units to pulse $\textbf{e}_{t}$, for $t=2,\ldots,T$. The usual randomization sampling results hold (Imbens and Rubin, 2015):

[TABLE]

where we defined:

[TABLE]

and

[TABLE]

where $\bar{Y}_{t}$ indicates averaging over all units. Occasionally, we will write $V_{h}^{(t)}(\underline{\textbf{Y}}),V_{0,e_{t}}(\underline{\textbf{Y}}),V_{1,e_{t}}(\underline{\textbf{Y}})$ to emphasize that these are functions of the potential outcomes schedule, $\underline{\textbf{Y}}$. Note also that the bias terms are zero because the estimators, $\hat{\lambda}_{t},\hat{\delta}_{t}$ are unbiased. Putting everything together, we obtain:

[TABLE]

Now is the key part of the argument. First, $V_{h}^{(t)}$ is a function only of $\textbf{y}_{t}(h)=(Y_{1t}(h),\ldots,Y_{Nt}(h))$, say, $V_{h}^{(t)}=V(\textbf{y}_{t}(h))$. Furthermore, $\textbf{y}_{t}(\textbf{h})$ takes values from $\mathcal{Y}$. Therefore,

[TABLE]

here, we allow $\operatorname*{argmax}$ to return a set. In particular, $\mathcal{Y}^{\mathrm{opt}}$ is a set because there might be many vectors that maximize $V$; the set can be a singleton, but it is not empty. Now, for every element $\textbf{y}\in\mathcal{Y}^{\mathrm{opt}}$, it holds

[TABLE]

since $\underline{\mathds{Y}}(\mathcal{Y})$ contains all potential outcomes schedules, such that the columns of every matrix in every schedule are from $\mathcal{Y}$. Take any vector $\textbf{y}\in\mathcal{Y}^{\mathrm{opt}}$. Define $\textbf{Y}^{\mathrm{opt}}$ as the $N\times T$ matrix where each column is equal to y, and define $\underline{\textbf{Y}}^{\mathrm{opt}}=[\textbf{Y}^{\mathrm{opt}},\ldots,\textbf{Y}^{\mathrm{opt}}]$ the potential outcomes schedule containing $T+1$ copies of $\textbf{Y}^{\mathrm{opt}}$. Then, for all $h=\mathbf{0},\mathbf{1},\textbf{e}_{t}$, and all $t=2,\ldots,T$,

[TABLE]

since the potential outcomes are bounded. It then follows that:

[TABLE]

since we can maximize $V_{1}^{(t)},V_{0}^{(t)},V_{e_{t}}^{(t)}$ separately for every $t$ by construction of the $\underline{\textbf{Y}}^{\mathrm{opt}}$ and the argument in Equation (21).

Now, we need to turn our attention to the negative terms in Equation (20). First, for all $t$,

[TABLE]

by definition of $\underline{\textbf{Y}}^{\mathrm{opt}}$. But we also have:

[TABLE]

because the $V$-functions are non-negative. It follows that

[TABLE]

We conclude that

[TABLE]

where we combined Equation (22) and Equation (23). ∎

A.2 Proof of Theorem 1

Proof.

If $\eta$ is minimax optimal, then by Lemma 2, $\tilde{\eta}$ is also minimax; our strategy, therefore, will be to find the design of the form $\tilde{\eta}$ that achieves the minimax risk. For a design $\tilde{\eta}$, we have, by Lemma 4,

[TABLE]

since $V^{\ast}$ does not depend on Z. It follows that $\min_{\eta}\max_{\underline{\textbf{Y}}\in\underline{\mathds{Y}}(\mathcal{Y})}r(\tilde{\eta}_{\textbf{Z}};\underline{\textbf{Y}})$ is attained by the design $\eta$ that satisfies:

[TABLE]

Let $\eta^{\mathrm{opt}}$ be a design satisfying Equation (24), and consider $\tilde{\eta}^{\mathrm{opt}}$. For any Z such that $\eta^{\mathrm{opt}}(\textbf{Z})>0$, we have:

[TABLE]

for any permutation $\pi\in\mathsf{S}_{N}$; that is, $\tilde{\eta}^{\mathrm{opt}}(\pi\cdot\textbf{Z})=\tilde{\eta}^{\mathrm{opt}}(\textbf{Z})=c$, for any permutation $\pi\in\mathsf{S}_{N}$. But the permutations of Z are the assignments such that:

[TABLE]

It is easy to verify that $\tilde{\eta}^{\mathrm{opt}}$ assigns mass zero to the assignments Z that do not satisfy Equation (25), and so in conclusion, $\tilde{\eta}^{\mathrm{opt}}$ is the completely randomized design with $N_{1}^{\mathrm{opt}}$, $N_{0}^{\mathrm{opt}}$ and $\{N_{e_{t}}^{\mathrm{opt}}\}$ satisfying Equation (25).

∎

A.3 Proof of Proposition 1

Proof.

The Lagrangian function corresponding to the objective function is:

[TABLE]

Setting the partial derivatives of the Lagrangian to zero yields the following system of equations:

[TABLE]

whose solution we write as a function of $\lambda$:

[TABLE]

But the total number of units assigned must add up to $N$, which allows us to solve for $\lambda$:

[TABLE]

leading to the following solution:

[TABLE]

This completes the proof. ∎

Appendix B Proof of results in Section 3.4

B.1 Proof of Theorem 2

The proof of Theorem 2 follows exactly the same lines as that of Theorem 4 which we prove below in Appendix D.

B.2 Proof of Proposition 2

Proof.

Consider the objective function:

[TABLE]

then notice that:

[TABLE]

Make $\frac{d\rho}{dN_{e_{t}}}=0$ and $\frac{d\rho}{dN_{e_{t-1}}}=0$, and the terms cancel between them by subtraction as follows:

[TABLE]

Therefore, we can write the same equation for $t=3,\ldots,T$

[TABLE]

Add all these equations on the left and right sides, we have

[TABLE]

Also, we have

[TABLE]

Plug these two equations to Equation 26, we have

[TABLE]

where $c_{T}=1$. Plug this to the $\frac{2}{(N_{e_{T}})^{2}}=\frac{2}{(N_{e_{T-1}})^{2}}-\frac{1}{[N_{0}+N_{e_{T}}]^{2}}$, we have

[TABLE]

where c_{T-1}=\bigg{[}\frac{1}{c_{T}^{2}}+\frac{1}{(1+\sqrt{2}c_{T})^{2}}\bigg{]}^{-1/2}. So, do the same operation for other equations, we have

[TABLE]

where

[TABLE]

Plug these $\{N_{e_{t}}\}$ to Equation 28, we have

[TABLE]

Solve this equation, we get

[TABLE]

Finally, we have

[TABLE]

∎

Appendix C Proof of results in Section 4.1

C.1 Proof of Theorem 3

Lemma 1 through Lemma 3 hold trivially with the weighted loss function. Adapting Lemma 4 requires more care.

Lemma 5** (Analog to Lemma 4).**

Let $\eta\in\mathds{H}$ and $\underline{\mathds{Y}}(\mathcal{Y})$ where $\mathcal{Y}$ is bounded and permutation-invariant, then:

[TABLE]

where $V^{\ast}$ is as in Lemma 4.

Proof of Lemma 5.

The proof requires to make some of same modifications made in the proof of Lemma 7. Here, we give a high-level view of some of the changes in the proof. We have:

[TABLE]

since the bias is zero under complete randomization. We therefore have (as in the proof of Lemma 4):

[TABLE]

and it follows that:

[TABLE]

which concludes the proof. ∎

Proof of Theorem 3.

The proof follows exactly the lines of the proof of Theorem 1 with the modified lemmas. ∎

C.2 Proof of Proposition 3

The proof is similar to that of Proposition 2, so we focus on the parts that change:

Proof of Proposition 3.

Let

[TABLE]

where we use $\phi$ instead of $\rho$ for the objective function, since $\rho$ is already used to denote the weight in this section. Now notice that:

[TABLE]

and

[TABLE]

and so:

[TABLE]

By telescoping the sums, we have

[TABLE]

In addition, we have:

[TABLE]

Combining with the previous equation, as in the proof of Proposition 2 yields:

[TABLE]

and so $N_{e_{T}}=N_{0}\ell c_{T}$ where $\ell=(1-\rho)^{-1/2}$ and $c_{T}=1$. Plugging this into the recursive definition of $N_{e_{t}}$, starting from $N_{e_{T}}$ we get

[TABLE]

which implies:

[TABLE]

where c_{T-1}=\bigg{[}\frac{1}{C_{T}^{2}}+\frac{1}{1+\ell c_{T}}\bigg{]}^{-1/2}. Now reasoning by recurrence on $t$, assume that $N_{e_{t^{\prime}}}=N_{0}\ell c_{t^{\prime}}$ with c_{t^{\prime}}=\bigg{[}\frac{1}{c_{t^{\prime}+1}^{2}}+\frac{1}{(1+\ell\sum_{t^{\prime\prime}>t^{\prime}}c_{t^{\prime\prime}})^{2}}\bigg{]}^{-1/2} for all $t^{\prime}\geq t$. We then have:

[TABLE]

and therefore:

[TABLE]

This proves that $N_{e_{t}}=N_{0}\ell c_{t}$ for all $t=2,\ldots,T$. Now since all units are assigned to exactly one arm, we have:

[TABLE]

The last step is to obtain an expression for $N_{0}$ as a function of $\{c_{t}\}_{t=2}^{T}$ and $\ell$. Notice that:

[TABLE]

which completes the proof. ∎

Appendix D Proof of results in Section 4.2

The proof of Theorem 4 follows the same lines as that of Theorem 1, with a few modifications. We first state and prove an analog to Lemma 1. Lemmas 2 and 3 carry through unchanged (they only depend on the invariance of the loss). We then prove a slightly modified version of Lemma 4. The proof of Theorem 4 follows directly from these modified lemmas. In this section, we use the loss function:

[TABLE]

where:

[TABLE]

with $N_{tk}=|\mathcal{C}_{k}^{(t)}|$. Under Assumption 3, $\hat{\beta}_{t}$ can be rewritten:

[TABLE]

We now state a slightly modified version of Lemma 1.

Lemma 6** (Analog to Lemma 1).**

Let $\pi\in\mathsf{S}_{N}$, Z and $\underline{\textbf{Y}}$. Then:

[TABLE]

Proof.

The only element that is changed from Lemma 1 is the use of $\hat{\beta}_{t}$ instead of $\hat{\delta}_{t}$. More specifically, under Assumption 3, if we let:

[TABLE]

and $\hat{\beta}_{t}=\hat{\beta}_{t}^{(\textbf{e}_{t})}(\textbf{Z},\underline{\textbf{Y}})-\hat{\beta}_{t}^{(k)}(\textbf{Z},\underline{\textbf{Y}})$, all we need to show is that $\hat{\beta}_{t}^{(k)}(\pi\cdot\textbf{Z},\pi\cdot\underline{\textbf{Y}})=\hat{\beta}_{t}^{(k)}(\textbf{Z},\underline{\textbf{Y}})$ for all $\pi$ and all $t=2,\ldots,T$, since this is the only new term.

First, notice that:

[TABLE]

This implies that $\mathds{1}(i\in\mathcal{C}_{k}^{(t)}(\pi\cdot\textbf{Z}))=\mathds{1}(\pi^{-1}(i)\in\mathcal{C}_{k}^{(t)})$. Moreover, since $\pi$ is a permutation, this also implies that:

[TABLE]

and therefore:

[TABLE]

∎

Lemma 7**.**

[Analogous to Lemma 4] Let $\eta\in\mathds{H}$ and $\underline{\mathds{Y}}(\mathcal{Y})$ where $\mathcal{Y}$ is bounded and permutation-invariant, then:

[TABLE]

where

[TABLE]

Proof.

We only discuss the modifications that need to be made to the proof of Lemma 4.

First, for $t$, define $\mathbf{0}^{\ast}=\{\mathbf{0}\text{ or }\textbf{e}_{t^{\prime}},\,\,t^{\prime}\in\mathcal{T}_{k}^{(t)}\}$. Since the treatment is completely randomized, then $\mathbf{0}^{\ast}$ is also completely randomized ($N_{tk}$ out of $N$). This means that $\text{bias}_{\tilde{\delta}_{\textbf{Z}}}(\hat{\beta}_{t},\delta_{t})=0$, and:

[TABLE]

The rest of the proof follows the lines of the proof of Lemma 4. ∎

Proof of Theorem 4.

The proof follows exactly the lines of the proof of Theorem 1, using Lemmas 6 and 7 instead of Lemmas 1 and 4. ∎

Appendix E Randomization-inference

In the randomization-based framework we adopt, the potential outcomes are considered fixed, and the only source of randomness is the random assignment Z; in particular, inference is performed with respect to the design $\eta$ used to randomize the assignment. The minimax optimal designs we obtain lend themselves to straightforward randomization-based inference since they are completely randomized experiments, and the estimator used are differences-in-means. In particular, if $\eta^{\mathrm{opt}}$ is a minimax optimal design as in Theorem 1, then $\text{Bias}_{\eta^{\mathrm{opt}}}(\hat{\lambda}_{t},\lambda_{t})=0$, and $\text{Bias}_{\eta^{\mathrm{opt}}}(\hat{\delta}_{t},\delta_{t})=0$, for $t=2,\ldots,T$. The usual variance formulas hold (Imbens and Rubin, 2015):

[TABLE]

where

[TABLE]

If $\eta^{\mathrm{opt}}$ is a minimax optimal design as in Theorem 2, similar results hold but with

[TABLE]

The variances of the estimators under the designs obtained in our other theorems can be obtained similarly. The standard conservative estimators of these variance elements can be obtained (Imbens and Rubin, 2015) and since $\hat{\lambda}_{t}$, $\hat{\delta}_{t}$ and $\hat{\gamma}_{t}$ are asymptotically normal under mild conditions, conservative confidence intervals can be constructed.

Appendix F Expected risk

Our theory predicts that that our minimax design will minimize the maximum risk –– the previous set of simulations illustrated the magnitude of the reduction. However, our theory does not offer any sort of guarantee on the expected risk under any specific outcome model. The simulations in this section aim to give some insights into how our designs perform in terms of expected risk under two simple outcome models.

• •

Standard: Consider the following model:

[TABLE]

where $\alpha_{i}$ and $\beta_{t}$ are fixed effects associated to unit $i$ and time period $t$, respectively. It can be verified that under this model, the expected value of the instantaneous effect is constant and equal to $\mu+\delta$. The term $\gamma_{Z_{i(t-1)}}$ captures the residual effect from the previous time step.

• •

Habituation: Consider the following model:

[TABLE]

where $\rho\in[0,1)$ represents a decay in treatment efficacy if the treatment is repeated between successive time periods.

We set the parameters values to $\mu=0$, $\alpha_{i}=\log(i)$, $\beta_{t}=\log(t)$, $\delta=1.0$, $\gamma=-1$, $\rho=0.5$, $\epsilon_{it}\sim\mathcal{N}(0,4^{2})$. The results presented below appear to be quite robust to different parameter specifications. 111We also tried $\alpha_{i}=\sqrt{i},\beta_{t}=\sqrt{t}$, and $\delta$ from 1 to 5 and $\gamma$ from -5 to -1, and their combinations, without seeing any qualitative change in the results. We consider settings with $N=100,200,500$ units, $T=10,15,20,25,30$ maximum time periods, and perform $100$ runs for every experimental setting.

Figure 3 displays the distribution of the risk values defined in Equation (7) for the minimax and BCRD designs. From the results of Figure 3, we see that the minimax design achieves, in general, smaller risk values than BCRD, especially as the number of units, $N$, increases. For fixed $N$, an increase in $T$ generally leads to an increase in expected risk values. This is expected since there are effectively fewer data to estimate the individual estimands, $\lambda_{t},\delta_{t}$ (the effect is more evident in the $N=100$ panel). This also explains why the variance for both designs decreases in general as $N$ increases.