An adaptive design for optimizing treatment assignment in randomized clinical trials

TL;DR

This paper introduces a multi-stage adaptive design for randomized clinical trials that improves treatment assignment efficiency by updating based on interim data, especially when prior variance information is limited.

Contribution

It proposes a practical multi-stage adaptive treatment assignment mechanism that enhances efficiency in clinical trials with limited prior variance data.

Findings

Simulation shows substantial efficiency gains over traditional designs.

The method is effective even with little prior information.

Illustrated with real stroke trial data.

Abstract

The treatment assignment mechanism in a randomized clinical trial can be optimized for statistical efficiency within a specified class of randomization mechanisms. Optimal designs of this type have been characterized in terms of the variances of potential outcomes conditional on baseline covariates. Approximating these optimal designs requires information about the conditional variance functions, which is often unavailable or unreliable at the design stage. As a practical solution to this dilemma, we propose a multi-stage adaptive design that allows the treatment assignment mechanism to be modified at interim analyses based on accruing information about the conditional variance functions. This adaptation has profound implications on the distribution of trial data, which need to be accounted for in treatment effect estimation. We consider a class of treatment effect estimators that are consistent and asymptotically normal, identify the most efficient estimator within this class, and approximate the most efficient estimator by substituting estimates of unknown quantities. Simulation results indicate that, when there is little or no prior information available, the proposed design can bring substantial efficiency gains over conventional one-stage designs based on the same prior information. The methodology is illustrated with real data from a completed trial in stroke.

Tables

Table 1. Table 1: Simulation-based relative efficiency results for simple and optimized estimators under various one-stage (1S) and two-stage (2S) designs (see Section 5 for details).

Setting	${\gamma }_1$	Design	Estimator	$𝑿$
				$W_1$	$W_2$	$W_3$	${(W_1,W_2)}^{\prime }$	${(W_1,W_3)}^{\prime }$	${(W_2,W_3)}^{\prime }$	$𝑾$
1	1	1S CIR	simple	$0.95$	$0.95$	$0.95$	$0.95$	$0.95$	$0.95$	$0.95$
			optimized	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$
		2S CIR	simple	$0.98$	$0.99$	$0.98$	$0.99$	$0.98$	$0.98$	$0.99$
			optimized	$1.11$	$1.12$	$1.12$	$1.12$	$1.12$	$1.13$	$1.14$
		2S CDR	simple	$1.05$	$1.03$	$1.05$	$1.03$	$1.03$	$0.98$	$0.91$
			optimized	$1.16$	$1.15$	$1.18$	$1.16$	$1.18$	$1.18$	$1.18$
	2	1S CIR	simple	$0.95$	$0.95$	$0.95$	$0.95$	$0.95$	$0.95$	$0.95$
			optimized	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$
		2S CIR	simple	$0.99$	$1.00$	$0.99$	$1.00$	$0.99$	$1.00$	$1.00$
			optimized	$1.12$	$1.13$	$1.13$	$1.13$	$1.13$	$1.14$	$1.15$
		2S CDR	simple	$1.05$	$1.02$	$1.03$	$1.03$	$1.01$	$0.97$	$0.88$
			optimized	$1.16$	$1.15$	$1.16$	$1.17$	$1.16$	$1.17$	$1.18$
2	1	1S CIR	simple	$0.87$	$0.87$	$0.88$	$0.87$	$0.86$	$0.86$	$0.84$
			optimized	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$
		1S CDR	simple	$0.81$	$0.82$	$0.81$	$0.77$	$0.75$	$0.65$	$0.42$
			optimized	$0.88$	$0.77$	$0.67$	$0.79$	$0.91$	$0.67$	$0.31$
		2S CIR	simple	$0.90$	$0.91$	$0.91$	$0.86$	$0.85$	$0.87$	$0.88$
			optimized	$1.04$	$1.06$	$1.05$	$1.01$	$1.01$	$1.04$	$1.05$
		2S CDR	simple	$0.87$	$0.86$	$0.90$	$0.83$	$0.82$	$0.71$	$0.57$
			optimized	$1.00$	$1.01$	$1.04$	$1.00$	$1.01$	$0.99$	$0.97$
		2S Hybrid	simple	$0.92$	$0.90$	$0.93$	$0.86$	$0.90$	$0.82$	$0.77$
			optimized	$1.05$	$1.06$	$1.09$	$1.01$	$1.06$	$1.05$	$1.09$
	2	1S CIR	simple	$0.91$	$0.87$	$0.88$	$0.88$	$0.87$	$0.87$	$0.85$
			optimized	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$
		1S CDR	simple	$0.81$	$0.81$	$0.78$	$0.74$	$0.74$	$0.66$	$0.45$
			optimized	$0.90$	$0.89$	$0.70$	$0.81$	$0.48$	$0.54$	$0.70$
		2S CIR	simple	$0.92$	$0.87$	$0.87$	$0.88$	$0.89$	$0.87$	$0.87$
			optimized	$1.01$	$1.01$	$1.02$	$1.03$	$1.06$	$1.05$	$1.03$
		2S CDR	simple	$0.84$	$0.83$	$0.84$	$0.81$	$0.82$	$0.75$	$0.57$
			optimized	$0.99$	$0.99$	$1.01$	$1.00$	$1.03$	$1.04$	$0.96$
		2S Hybrid	simple	$0.89$	$0.86$	$0.87$	$0.89$	$0.92$	$0.85$	$0.73$
			optimized	$1.02$	$1.02$	$1.02$	$1.05$	$1.09$	$1.09$	$1.06$

Table 2. Table 2: Simulation-based coverage results (at nominal level 95%) for optimized estimators under various two-stage (2S) designs (see Section 5 for details).

Setting	${\gamma }_1$	Design	$𝑿$
			$W_1$	$W_2$	$W_3$	${(W_1,W_2)}^{\prime }$	${(W_1,W_3)}^{\prime }$	${(W_2,W_3)}^{\prime }$	$𝑾$
1	1	2S CIR	$0.949$	$0.950$	$0.951$	$0.952$	$0.952$	$0.951$	$0.954$
		2S CDR	$0.957$	$0.956$	$0.954$	$0.956$	$0.959$	$0.957$	$0.958$
	2	2S CIR	$0.951$	$0.949$	$0.951$	$0.950$	$0.951$	$0.952$	$0.951$
		2S CDR	$0.953$	$0.952$	$0.951$	$0.956$	$0.954$	$0.955$	$0.957$
2	1	2S CIR	$0.953$	$0.954$	$0.951$	$0.947$	$0.949$	$0.956$	$0.951$
		2S CDR	$0.944$	$0.951$	$0.950$	$0.954$	$0.951$	$0.952$	$0.950$
		2S Hybrid	$0.955$	$0.953$	$0.953$	$0.950$	$0.952$	$0.953$	$0.958$
	2	2S CIR	$0.948$	$0.953$	$0.952$	$0.949$	$0.949$	$0.952$	$0.951$
		2S CDR	$0.948$	$0.953$	$0.955$	$0.954$	$0.956$	$0.953$	$0.946$
		2S Hybrid	$0.953$	$0.953$	$0.955$	$0.954$	$0.952$	$0.957$	$0.954$

Table 3. Table 3: Results of re-designing and re-analyzing the NINDS trial: point estimates (standard errors; relative efficiency) of the re-scaled mean score difference for various choices of Y Y and 𝑿 \boldsymbol{X} , under the original one-stage (1S) 1:1 CIR and proposed two-stage (2S) CIR and CDR designs.

$𝑿$	1S CIR		2S CIR		2S CDR
$Y=\text{Barthel Index}$
age	0.096 (0.029; ref)		0.125 (0.029; 1.03)		0.136 (0.028; 1.06)
NIHSS	0.096 (0.029; ref)		0.134 (0.029; 1.04)		0.092 (0.028; 1.14)
diabetes	0.096 (0.029; ref)		0.115 (0.029; 1.04)		0.109 (0.028; 1.11)
age, NIHSS	0.096 (0.029; ref)		0.079 (0.029; 1.03)		0.110 (0.029; 1.02)
age, diabetes	0.096 (0.029; ref)		0.107 (0.029; 1.06)		0.119 (0.029; 1.00)
NIHSS, diabetes	0.096 (0.029; ref)		0.103 (0.029; 1.05)		0.126 (0.029; 1.04)
age, NIHSS, diabetes	0.096 (0.029; ref)		0.088 (0.028; 1.07)		0.138 (0.029; 1.04)
$Y=\text{Modified Rankin Scale}$
age	0.095 (0.024; ref)		0.106 (0.024; 1.02)		0.117 (0.024; 1.00)
NIHSS	0.095 (0.024; ref)		0.115 (0.024; 1.01)		0.116 (0.023; 1.13)
diabetes	0.095 (0.024; ref)		0.132 (0.024; 1.03)		0.118 (0.024; 1.01)
age, NIHSS	0.095 (0.024; ref)		0.084 (0.024; 1.03)		0.095 (0.024; 1.06)
age, diabetes	0.095 (0.024; ref)		0.107 (0.024; 1.01)		0.136 (0.024; 1.04)
NIHSS, diabetes	0.095 (0.024; ref)		0.117 (0.024; 1.05)		0.124 (0.024; 1.03)
age, NIHSS, diabetes	0.095 (0.024; ref)		0.096 (0.023; 1.07)		0.098 (0.024; 1.00)
$Y=\text{Glasgow Outcome Scale}$
age	0.066 (0.021; ref)		0.089 (0.021; 1.01)		0.082 (0.021; 1.05)
NIHSS	0.066 (0.021; ref)		0.070 (0.021; 1.01)		0.087 (0.021; 1.04)
diabetes	0.066 (0.021; ref)		0.088 (0.021; 1.02)		0.087 (0.021; 1.04)
age, NIHSS	0.066 (0.021; ref)		0.078 (0.021; 1.01)		0.072 (0.020; 1.11)
age, diabetes	0.066 (0.021; ref)		0.092 (0.020; 1.09)		0.090 (0.021; 1.03)
NIHSS, diabetes	0.066 (0.021; ref)		0.075 (0.020; 1.11)		0.111 (0.021; 1.02)
age, NIHSS, diabetes	0.066 (0.021; ref)		0.079 (0.020; 1.05)		0.089 (0.021; 1.01)
$Y=\text{NIHSS}$
age	0.060 (0.027; ref)		0.067 (0.026; 1.04)		0.066 (0.027; 0.98)
NIHSS	0.060 (0.027; ref)		0.059 (0.024; 1.18)		0.106 (0.025; 1.13)
diabetes	0.060 (0.027; ref)		0.035 (0.025; 1.09)		0.076 (0.026; 1.08)
age, NIHSS	0.060 (0.027; ref)		0.079 (0.026; 1.06)		0.114 (0.026; 1.05)
age, diabetes	0.060 (0.027; ref)		0.081 (0.027; 0.98)		0.097 (0.026; 1.07)
NIHSS, diabetes	0.060 (0.027; ref)		0.079 (0.026; 1.01)		0.064 (0.026; 1.07)
age, NIHSS, diabetes	0.060 (0.027; ref)		0.044 (0.026; 1.06)		0.065 (0.026; 1.03)

Table 4. Table S1: Additional simulation results: (standard deviation; median standard error) for optimized estimators under various two-stage (2S) designs.

Setting	${\gamma }_1$	Design	$𝑿$
			$W_1$	$W_2$	$W_3$	${(W_1,W_2)}^{\prime }$	${(W_1,W_3)}^{\prime }$	${(W_2,W_3)}^{\prime }$	$𝑾$
1	1	2S CIR	(0.253, 0.253)	(0.252, 0.253)	(0.252, 0.252)	(0.252, 0.252)	(0.252, 0.252)	(0.251, 0.252)	(0.250, 0.252)
		2S CDR	(0.248, 0.253)	(0.250, 0.253)	(0.246, 0.252)	(0.248, 0.252)	(0.246, 0.251)	(0.246, 0.251)	(0.246, 0.250)
	2	2S CIR	(0.249, 0.248)	(0.248, 0.248)	(0.248, 0.248)	(0.248, 0.248)	(0.248, 0.248)	(0.247, 0.248)	(0.247,0.248)
		2S CDR	(0.245, 0.248)	(0.246, 0.248)	(0.245, 0.248)	(0.244, 0.248)	(0.245, 0.247)	(0.244, 0.247)	(0.243, 0.246)
2	1	2S CIR	(0.246, 0.245)	(0.243, 0.245)	(0.246, 0.244)	(0.247, 0.244)	(0.249, 0.244)	(0.245, 0.244)	(0.245, 0.243)
		2S CDR	(0.251, 0.246)	(0.248, 0.246)	(0.247, 0.244)	(0.248, 0.246)	(0.249, 0.243)	(0.251, 0.243)	(0.255, 0.242)
		2S Hybrid	(0.245, 0.245)	(0.243, 0.245)	(0.242, 0.244)	(0.247, 0.244)	(0.243, 0.242)	(0.244, 0.242)	(0.240, 0.241)
	2	2S CIR	(0.242, 0.240)	(0.239, 0.239)	(0.241, 0.239)	(0.239, 0.238)	(0.240, 0.239)	(0.237, 0.238)	(0.242, 0.238)
		2S CDR	(0.245, 0.241)	(0.241, 0.240)	(0.242, 0.239)	(0.243, 0.241)	(0.244, 0.239)	(0.238, 0.238)	(0.250, 0.237)
		2S Hybrid	(0.241, 0.240)	(0.239, 0.239)	(0.240, 0.239)	(0.237, 0.238)	(0.237, 0.238)	(0.233, 0.237)	(0.239, 0.236)

Table 5. Table S2: Simulation-based relative efficiency results for simple and optimized estimators under various one-stage (1S) and two-stage (2S) designs with n 1 = n 0 = 150 n_{1}=n_{0}=150 (see Appendix D for details).

Setting	${\gamma }_1$	Design	Estimator	$𝑿$
				$W_1$	$W_2$	$W_3$	${(W_1,W_2)}^{\prime }$	${(W_1,W_3)}^{\prime }$	${(W_2,W_3)}^{\prime }$	$𝑾$
1	1	1S CIR	simple	$0.94$	$0.95$	$0.94$	$0.94$	$0.95$	$0.95$	$0.95$
			optimized	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$
		2S CIR	simple	$1.00$	$1.02$	$1.01$	$0.99$	$1.03$	$1.00$	$0.99$
			optimized	$1.13$	$1.14$	$1.12$	$1.08$	$1.14$	$1.13$	$1.15$
		2S CDR	simple	$1.01$	$1.02$	$1.00$	$0.97$	$1.00$	$0.95$	$0.84$
			optimized	$1.11$	$1.13$	$1.13$	$1.08$	$1.13$	$1.15$	$1.10$
	2	1S CIR	simple	$0.95$	$0.95$	$0.95$	$0.96$	$0.95$	$0.95$	$0.95$
			optimized	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$
		2S CIR	simple	$1.01$	$1.01$	$1.02$	$0.99$	$1.04$	$0.98$	$1.02$
			optimized	$1.16$	$1.15$	$1.15$	$1.11$	$1.17$	$1.12$	$1.17$
		2S CDR	simple	$1.06$	$1.04$	$1.02$	$0.99$	$1.01$	$0.90$	$0.83$
			optimized	$1.16$	$1.17$	$1.15$	$1.12$	$1.15$	$1.09$	$1.07$
2	1	1S CIR	simple	$0.90$	$0.88$	$0.87$	$0.88$	$0.89$	$0.85$	$0.85$
			optimized	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$
		1S CDR	simple	$0.86$	$0.83$	$0.81$	$0.81$	$0.79$	$0.67$	$0.47$
			optimized	$0.87$	$0.57$	$0.77$	$0.87$	$0.90$	$0.58$	$0.52$
		2S CIR	simple	$0.91$	$0.91$	$0.87$	$0.92$	$0.90$	$0.86$	$0.85$
			optimized	$1.05$	$1.07$	$1.02$	$1.06$	$1.06$	$1.01$	$1.04$
		2S CDR	simple	$0.87$	$0.87$	$0.86$	$0.91$	$0.84$	$0.74$	$0.60$
			optimized	$1.01$	$1.03$	$1.03$	$1.09$	$1.04$	$0.99$	$0.72$
		2S Hybrid	simple	$0.89$	$0.91$	$0.87$	$0.88$	$0.90$	$0.85$	$0.75$
			optimized	$1.04$	$1.07$	$1.02$	$1.09$	$1.12$	$1.08$	$1.05$
	2	1S CIR	simple	$0.90$	$0.89$	$0.87$	$0.86$	$0.85$	$0.86$	$0.85$
			optimized	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$
		1S CDR	simple	$0.88$	$0.86$	$0.82$	$0.72$	$0.78$	$0.63$	$0.49$
			optimized	$0.95$	$0.74$	$0.93$	$0.46$	$0.48$	$0.80$	$0.60$
		2S CIR	simple	$0.91$	$0.94$	$0.89$	$0.86$	$0.86$	$0.87$	$0.85$
			optimized	$1.05$	$1.09$	$1.05$	$1.03$	$1.00$	$1.05$	$1.03$
		2S CDR	simple	$0.90$	$0.90$	$0.87$	$0.80$	$0.80$	$0.70$	$0.59$
			optimized	$1.05$	$1.08$	$1.04$	$1.01$	$1.00$	$0.95$	$0.93$
		2S Hybrid	simple	$0.96$	$0.89$	$0.91$	$0.86$	$0.86$	$0.80$	$0.76$
			optimized	$1.10$	$1.05$	$1.06$	$1.02$	$1.05$	$1.05$	$1.06$

Table 6. Table S3: Simulation-based relative efficiency for simple and optimized estimators under one-stage (1S) and two-stage (2S) designs with various stage 1 sample size proportions ( n 1 / n n_{1}/n ) in Setting 1 (see Appendix D for details).

$n_1/n$	${\gamma }_1$	Design	Estimator	$𝑿$
				$W_1$	$W_2$	$W_3$	${(W_1,W_2)}^{\prime }$	${(W_1,W_3)}^{\prime }$	${(W_2,W_3)}^{\prime }$	$𝑾$
0.3	1	1S CIR	simple	$0.95$	$0.95$	$0.95$	$0.95$	$0.94$	$0.95$	$0.94$
			optimized	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$
		2S CIR	simple	$1.03$	$1.04$	$0.96$	$1.01$	$1.00$	$0.98$	$0.96$
			optimized	$1.14$	$1.16$	$1.08$	$1.14$	$1.14$	$1.12$	$1.11$
		2S CDR	simple	$0.99$	$0.98$	$0.96$	$0.95$	$0.96$	$0.90$	$0.69$
			optimized	$1.11$	$1.12$	$1.10$	$1.12$	$1.10$	$1.14$	$1.04$
	2	1S CIR	simple	$0.96$	$0.95$	$0.95$	$0.96$	$0.94$	$0.94$	$0.94$
			optimized	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$
		2S CIR	simple	$1.00$	$1.02$	$1.02$	$1.01$	$0.99$	$1.02$	$1.05$
			optimized	$1.12$	$1.13$	$1.15$	$1.15$	$1.13$	$1.16$	$1.23$
		2S CDR	simple	$0.98$	$0.98$	$1.01$	$0.94$	$0.93$	$0.84$	$0.76$
			optimized	$1.07$	$1.12$	$1.14$	$1.08$	$1.09$	$1.09$	$1.08$
0.7	1	1S CIR	simple	$0.95$	$0.95$	$0.95$	$0.95$	$0.95$	$0.95$	$0.95$
			optimized	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$
		2S CIR	simple	$1.00$	$0.99$	$0.98$	$1.00$	$0.99$	$0.99$	$0.99$
			optimized	$1.11$	$1.11$	$1.09$	$1.11$	$1.11$	$1.11$	$1.11$
		2S CDR	simple	$0.99$	$0.99$	$0.99$	$0.98$	$0.98$	$0.96$	$0.93$
			optimized	$1.11$	$1.10$	$1.10$	$1.12$	$1.11$	$1.11$	$1.12$
	2	1S CIR	simple	$0.94$	$0.94$	$0.94$	$0.95$	$0.95$	$0.95$	$0.95$
			optimized	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$
		2S CIR	simple	$1.01$	$1.01$	$1.01$	$0.99$	$0.99$	$0.98$	$0.98$
			optimized	$1.11$	$1.11$	$1.11$	$1.10$	$1.10$	$1.11$	$1.11$
		2S CDR	simple	$1.00$	$1.01$	$1.01$	$0.98$	$0.98$	$0.95$	$0.92$
			optimized	$1.10$	$1.11$	$1.11$	$1.10$	$1.11$	$1.10$	$1.12$

Table 7. Table S4: Simulation-based relative efficiency for simple and optimized estimators under one-stage (1S) and two-stage (2S) designs with various numbers of prognostic covariates (#prog) (see Appendix D for details).

#prog	${\gamma }_1$	Design	Estimator	$𝑿$
				$W_1$	$W_2$	$W_3$	${(W_1,W_2)}^{\prime }$	${(W_1,W_3)}^{\prime }$	${(W_2,W_3)}^{\prime }$	$𝑾$
1	1	1S CIR	simple	$0.97$	$0.97$	$0.97$	$0.97$	$0.97$	$0.97$	$0.97$
			optimized	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$
		2S CIR	simple	$1.01$	$1.01$	$1.00$	$1.01$	$1.00$	$1.00$	$1.00$
			optimized	$1.08$	$1.09$	$1.09$	$1.09$	$1.09$	$1.09$	$1.09$
		2S CDR	simple	$1.00$	$1.01$	$0.99$	$1.00$	$0.99$	$0.99$	$0.98$
			optimized	$1.08$	$1.09$	$1.12$	$1.08$	$1.11$	$1.11$	$1.11$
	2	1S CIR	simple	$0.98$	$0.98$	$0.98$	$0.98$	$0.98$	$0.98$	$0.98$
			optimized	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$
		2S CIR	simple	$0.99$	$0.99$	$0.99$	$0.99$	$0.99$	$1.00$	$1.00$
			optimized	$1.06$	$1.06$	$1.08$	$1.07$	$1.08$	$1.08$	$1.08$
		2S CDR	simple	$0.99$	$1.00$	$0.98$	$0.99$	$0.97$	$0.98$	$0.96$
			optimized	$1.06$	$1.08$	$1.11$	$1.06$	$1.10$	$1.11$	$1.09$
2	1	1S CIR	simple	$0.94$	$0.94$	$0.94$	$0.94$	$0.94$	$0.94$	$0.94$
			optimized	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$
		2S CIR	simple	$0.97$	$0.96$	$0.96$	$0.96$	$0.96$	$0.94$	$0.95$
			optimized	$1.08$	$1.09$	$1.09$	$1.09$	$1.08$	$1.08$	$1.09$
		2S CDR	simple	$0.98$	$0.98$	$0.98$	$0.97$	$0.97$	$0.89$	$0.88$
			optimized	$1.09$	$1.12$	$1.13$	$1.10$	$1.11$	$1.14$	$1.12$
	2	1S CIR	simple	$0.94$	$0.94$	$0.94$	$0.94$	$0.94$	$0.94$	$0.94$
			optimized	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$
		2S CIR	simple	$0.97$	$0.97$	$0.96$	$0.97$	$0.97$	$0.95$	$0.95$
			optimized	$1.08$	$1.09$	$1.08$	$1.08$	$1.08$	$1.08$	$1.08$
		2S CDR	simple	$0.97$	$0.98$	$0.97$	$0.97$	$0.96$	$0.90$	$0.90$
			optimized	$1.10$	$1.12$	$1.12$	$1.10$	$1.10$	$1.14$	$1.13$

Table 8. Table S5: Simulation-based relative efficiency results for simple and optimized estimators under various one-stage (1S) and two-stage (2S) designs, under a severely misspecified outcome mean model (see Appendix D for details).

Setting	${\gamma }_1$	Design	Estimator	$𝑿$
				$W_1$	$W_2$	$W_3$	${(W_1,W_2)}^{\prime }$	${(W_1,W_3)}^{\prime }$	${(W_2,W_3)}^{\prime }$	$𝑾$
1	1	1S CIR	simple	$0.95$	$0.95$	$0.95$	$0.95$	$0.95$	$0.95$	$0.95$
			optimized	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$
		2S CIR	simple	$0.99$	$0.99$	$0.98$	$0.99$	$0.98$	$0.99$	$0.99$
			optimized	$1.12$	$1.12$	$1.11$	$1.12$	$1.11$	$1.12$	$1.12$
		2S CDR	simple	$1.03$	$1.03$	$1.02$	$1.02$	$1.01$	$0.99$	$0.98$
			optimized	$1.13$	$1.14$	$1.13$	$1.13$	$1.12$	$1.11$	$1.10$
	2	1S CIR	simple	$0.95$	$0.95$	$0.95$	$0.95$	$0.95$	$0.95$	$0.95$
			optimized	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$	$1.00$
		2S CIR	simple	$0.99$	$0.99$	$0.99$	$1.00$	$0.99$	$1.00$	$0.99$
			optimized	$1.12$	$1.13$	$1.13$	$1.13$	$1.12$	$1.13$	$1.13$
		2S CDR	simple	$1.02$	$1.02$	$1.02$	$1.02$	$1.00$	$0.99$	$0.97$
			optimized	$1.14$	$1.14$	$1.13$	$1.14$	$1.12$	$1.12$	$1.10$

Equations

$$P\{A=1|\mathbf{W},Y(1),Y(0)\}=P(A=1)=\pi \in (0,1),$$

$$P\{A=1|\mathbf{W},Y(1),Y(0)\}=P(A=1|\mathbf{W})=p(\mathbf{W})\in (0,1),$$

$${\pi }_{\text{opt}}=\frac{g^{\prime }({\mu }_1)[E\{v_1(\mathbf{W})\}{]}^{1/2}}{g^{\prime }({\mu }_1)[E\{v_1(\mathbf{W})\}{]}^{1/2}+g^{\prime }({\mu }_0)[E\{v_0(\mathbf{W})\}{]}^{1/2}},$$

$$p_{\text{opt}}(\mathbf{W})=\frac{g^{\prime }({\mu }_1)v_1(\mathbf{W}{)}^{1/2}}{g^{\prime }({\mu }_1)v_1(\mathbf{W}{)}^{1/2}+g^{\prime }({\mu }_0)v_0(\mathbf{W}{)}^{1/2}}.$$

$${\hat{\mu }}_a^{(s)}=\frac{{\hat{E}}_s\{I(A_s=a)Y_s\}}{{\hat{E}}_s\{I(A_s=a)\}},s=1,2,$$

$${\hat{\delta }}^{\text{aug}}(b_1,b_2,\theta )=\hat{\delta }(\theta )-{\hat{E}}_1\left\{(A_1-{\pi }_1)b_1(\mathbf{W})\right\}-{\hat{E}}_2\left\{(A_2-{\pi }_2)b_2(\mathbf{W})\right\},$$

$$\sqrt{n_1}\left\{{\hat{\delta }}^{\text{aug}}(b_1,b_2,\theta )-\delta \right\}\stackrel{d}{⟶}N\left(0,{\sigma }_{\text{textsc}cir}^2(b_1,b_2,\theta )\right),$$

$${\sigma }_{\text{textsc}cir}^2(b_1,b_2,\theta )$$

$${\psi }_1^{\text{aug}}({\mathbf{O}}_1)$$

$${\psi }_2^{\text{aug}}({\mathbf{O}}_2^{\ast })$$

$$\begin{array}{c}P\{A_2^{\ast }=1|\mathbf{W},Y(1),Y(0)\}=P(A_2^{\ast }=1)={\pi }_2^{\ast },\\ Y_2^{\ast }=Y(A_2^{\ast })=A_2^{\ast }Y(1)+(1-A_2^{\ast })Y(0).\end{array}$$

$$b_{1,\text{opt}}(\mathbf{W};\theta )$$

$$b_{2,\text{opt}}(\mathbf{W};\theta )$$

$${\hat{b}}_1(\mathbf{W};\theta )$$

$${\hat{b}}_2(\mathbf{W};\theta )$$

$$b_1^{\ast }(\mathbf{W};\theta )$$

$$b_2^{\ast }(\mathbf{W};\theta )$$

$${\sigma }_{1,\text{textsc}cir}^{\ast 2}$$

$$\displaystyle\qquad\quad+g^{\prime}(\mu_{1})\left\{m_{1}^{*}(\boldsymbol{W})-\mu_{1}\right\}-g^{\prime}(\mu_{0})\left\{m_{0}^{*}(\boldsymbol{W})-\mu_{0}\right\}\bigg{]},$$

$${\sigma }_{2,\text{textsc}cir}^{\ast 2}$$

$$\displaystyle\qquad\quad+g^{\prime}(\mu_{1})\left\{m_{1}^{*}(\boldsymbol{W})-\mu_{1}\right\}-g^{\prime}(\mu_{0})\left\{m_{0}^{*}(\boldsymbol{W})-\mu_{0}\right\}\bigg{]}.$$

$${\hat{\sigma }}_{s,\text{textsc}cir}^2$$

$$\displaystyle\qquad\qquad+g^{\prime}(\widehat{\mu}_{1})\left\{\widehat{m}_{1}(\boldsymbol{W})-\widehat{\mu}_{1}\right\}-g^{\prime}(\widehat{\mu}_{0})\left\{\widehat{m}_{0}(\boldsymbol{W})-\widehat{\mu}_{0}\right\}\bigg{]},$$

$$\sqrt{n_1}\left\{{\hat{\delta }}^{\text{aug}}({\tilde{b}}_1,{\tilde{b}}_2,\tilde{\theta })-\delta \right\}\stackrel{d}{⟶}N\left(0,{\sigma }_{\text{textsc}cir}^2(b_1,b_2,\theta )\right);$$

$$\displaystyle\widehat{\operatorname{var}}_{1}\bigg{\{}\frac{\widehat{\theta}g^{\prime}(\widehat{\mu}_{1})A_{1}(Y_{1}-\widehat{\mu}_{1})}{\pi_{1}}-\frac{\widehat{\theta}g^{\prime}(\widehat{\mu}_{0})(1-A_{1})(Y_{1}-\widehat{\mu}_{0})}{1-\pi_{1}}-(A_{1}-\pi_{1})\widehat{b}_{1}(\boldsymbol{W})\bigg{\}}$$

$$\displaystyle+\frac{n_{1}}{n_{2}}\widehat{\operatorname{var}}_{2}\bigg{\{}\frac{(1-\widehat{\theta})g^{\prime}(\widehat{\mu}_{1})A_{2}(Y_{2}-\widehat{\mu}_{1})}{\pi_{2}}-\frac{(1-\widehat{\theta})g^{\prime}(\widehat{\mu}_{0})(1-A_{2})(Y_{2}-\widehat{\mu}_{0})}{1-\pi_{2}}-(A_{2}-\pi_{2})\widehat{b}_{2}(\boldsymbol{W})\bigg{\}}.$$

$$\operatorname{logit}\{\operatorname{P}(Y_{1}=1|A_{1},\boldsymbol{X})\}=\alpha_{0}+\alpha_{1}A_{1}+\mbox{\boldmath${\alpha}$}_{2}^{\prime}\boldsymbol{X}+\mbox{\boldmath${\alpha}$}_{3}^{\prime}(A_{1}\boldsymbol{X}).$$

$$\widehat{\mu}_{a}^{\text{ipw}(s)}=\frac{\widehat{\operatorname{E}}_{s}\Big{(}I(A_{s}=a)Y_{s}\big{/}\big{[}p_{s}(\boldsymbol{W})^{a}\{1-p_{s}(\boldsymbol{W})\}^{1-a}\big{]}\Big{)}}{\widehat{\operatorname{E}}_{s}\Big{(}I(A_{s}=a)\big{/}\big{[}p_{s}(\boldsymbol{W})^{a}\{1-p_{s}(\boldsymbol{W})\}^{1-a}\big{]}\Big{)}},\qquad s=1,2,\qquad a=0,1.$$

$${\hat{\delta }}^{\text{aipw}}(c_1,c_2,\eta )={\hat{\delta }}^{\text{ipw}}(\eta )-{\hat{E}}_1\left[\{A_1-p_1(\mathbf{W})\}{c}_1(\mathbf{W})\right]-{\hat{E}}_2\left[\{A_2-p_2(\mathbf{W})\}{c}_2(\mathbf{W})\right],$$

$$\sqrt{n_1}\left\{{\hat{\delta }}^{\text{aipw}}(c_1,c_2,\eta )-\delta \right\}\stackrel{d}{⟶}N\left(0,{\sigma }_{\text{textsc}cdr}^2(c_1,c_2,\eta )\right),$$

$${\sigma }_{\text{textsc}cdr}^2(c_1,c_2,\eta )$$

$${\psi }_1^{\text{aipw}}({\mathbf{O}}_1)$$

$${\psi }_2^{\text{aipw}}({\mathbf{O}}_2^{\ast })$$

$$-\{A_2^{\ast }-p_2^{\ast }(\mathbf{W})\}{c}_2(\mathbf{W}),$$

$$\begin{array}{c}P\{A_2^{\ast }=1|\mathbf{W},Y(1),Y(0)\}=P(A_2^{\ast }=1|\mathbf{W})=p_2^{\ast }(\mathbf{W}),\\ Y_2^{\ast }=Y(A_2^{\ast })=A_2^{\ast }Y(1)+(1-A_2^{\ast })Y(0).\end{array}$$

$$c_{1,\text{opt}}(\mathbf{W};\eta )$$

$$c_{2,\text{opt}}(\mathbf{W};\eta )$$

Full text

An Adaptive Design for Optimizing Treatment Assignment in Randomized Clinical Trials

Wei Zhang1, Zhiwei Zhang2,∗, and Aiyi Liu3

1State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China

2Biostatistics Innovation Group, Gilead Sciences, Foster City, California, USA

3Biostatistics and Bioinformatics Branch, Division of Population Health Research, Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health, Bethesda, Maryland, USA

∗Zhiwei.Zhang6@gilead.com

Abstract

The treatment assignment mechanism in a randomized clinical trial can be optimized for statistical efficiency within a specified class of randomization mechanisms. Optimal designs of this type have been characterized in terms of the variances of potential outcomes conditional on baseline covariates. Approximating these optimal designs requires information about the conditional variance functions, which is often unavailable or unreliable at the design stage. As a practical solution to this dilemma, we propose a multi-stage adaptive design that allows the treatment assignment mechanism to be modified at interim analyses based on accruing information about the conditional variance functions. This adaptation has profound implications on the distribution of trial data, which need to be accounted for in treatment effect estimation. We consider a class of treatment effect estimators that are consistent and asymptotically normal, identify the most efficient estimator within this class, and approximate the most efficient estimator by substituting estimates of unknown quantities. Simulation results indicate that, when there is little or no prior information available, the proposed design can bring substantial efficiency gains over conventional one-stage designs based on the same prior information. The methodology is illustrated with real data from a completed trial in stroke.

Key words: augmentation; covariate adjustment; interim analysis; optimal design; propensity score; treatment allocation

1 Introduction

Randomized clinical trials (RCTs) are widely considered the gold standard for comparing an experimental treatment with a control treatment. RCTs can be highly expensive in terms of time and money, making it crucial to optimize clinical trial design and analysis for statistical efficiency. The analysis of trial data can be made more efficient by incorporating information from baseline covariates that are related to clinical outcomes, a practice commonly known as covariate adjustment. There are many statistical methods for covariate adjustment (e.g., Tsiatis et al., 2008; Zhang et al., 2008; Moore and van der Laan, 2009; Wang et al., 2019; Zhang and Ma, 2019; Ye et al., 2023; Bannick et al., 2025), which provide covariate-adjusted treatment effect estimators that are asymptotically unbiased and generally more efficient than unadjusted estimators. Covariate adjustment has drawn a great deal of attention and serious consideration from practitioners and regulators (FDA, 2023).

Treatment assignment is a key aspect of trial design that can be optimized for statistical efficiency. For unadjusted treatment effect estimators based on treatment-specific sample means, the optimal allocation, known as the Neyman allocation (Neyman, 1934), assigns group sizes in proportion to the standard deviation of the primary outcome within each treatment group. Such estimators are generally inefficient as they do not make use of baseline covariate information. Zhang et al. (2023) derive optimal allocation ratios for covariate-adjusted treatment effect estimators, including the optimal one that attains the nonparametric information bound. The optimal allocation ratio for the optimal estimator can be regarded as an optimal design that maximizes the nonparametric information bound. In addition to conventional covariate-independent randomization (CIR), Zhang et al. (2023) also consider covariate-dependent randomization (CDR), which resembles observational studies except that the propensity score is specified and known by the investigator. The propensity score in CDR can be optimized for (locally) efficient treatment effect estimators available from the causal inference literature (e.g., van der Laan and Robins, 2003; van der Laan and Rose, 2011). The optimal propensity score for the efficient estimator represents the optimal CDR design that maximizes the nonparametric information bound over all CDR designs (including CIR designs as special cases). The optimal CIR and CDR designs differ fundamentally from covariate-adaptive randomization designs (e.g., Rosenberger and Sverdlov, 2008), which improve efficiency for individual estimators but do not change the nonparametric information bound relative to simple randomization (Rafi, 2023).

The optimal CIR and CDR designs are characterized using the variances of potential outcomes conditional on baseline covariates. Approximating these optimal designs requires preliminary information on the conditional variance functions. In practice, preliminary estimates of the conditional variance functions are often unavailable or unreliable, making it difficult to construct efficient designs based on the optimality results of Zhang et al. (2023). Their simulation results indicate that unreliable estimates of conditional variance functions can result in ill-optimized designs that perform poorly, especially when the optimal CDR design is targeted.

As a concrete example, consider the National Institute of Neurological Disorders and Stroke (NINDS) trial of recombinant tissue plasminogen activator (rt-PA) for treating acute ischemic stroke (NINDS rt-PA Stroke Study Group, 1995). In this trial, a total of 624 eligible patients were randomized 1:1 to receive rt-PA or placebo within 3 hours of stroke onset. Patients were evaluated at baseline (before randomization) and at 24 hours, 3 months, and 1 year post-treatment. The primary efficacy analysis found statistically significant improvements in clinical outcomes at 3 months. Zhang et al. (2023) show that, in this setting, optimal designs based on three dichotomous baseline covariates can reduce the variance by as much as 25% in treatment effect estimation. However, this improvement is based on the assumption that the true conditional variance functions are known at the design stage. This assumption is unlikely to hold in practice, which begs the question: what can be done to improve design efficiency (in terms of treatment assignment) with little or no information available about the conditional variance functions?

In this article, we propose an adaptive design as a practical answer to the above question. The proposed design consists of multiple stages separated by one or more interim analyses. In the first stage, treatment assignment may follow the standard $1:1$ CIR design (if no prior information is available) or a cautiously optimized CIR or CDR design based on prior information. At each interim analysis, the conditional variance functions are (re-)estimated with updated data, and the updated estimates are used to optimize treatment assignment in the next stage. In general, as information accrues over time, treatment assignment optimization can become more ambitious in terms of the set of covariates to consider, the amount of coarsening to apply (to the covariates), and the class of designs (CIR versus CDR) to optimize over.

While conceptually simple and intuitive, the proposed adaptive design does present serious analytical challenges in treatment effect estimation and inference. Because the treatment assignment mechanism may change over time, individual patient data are generally not identically distributed between different stages. Because the treatment assignment mechanism in a later stage may depend on data from previous stages, independence between patients is no longer guaranteed except within the first stage. We consider a class of treatment effect estimators indexed by a set of weights (for combining data across stages) and a set of augmentation functions (for incorporating covariate information). Each estimator in this class is consistent and asymptotically normal, and its asymptotic variance depends on the weights and augmentation functions. The optimal weights and augmentation functions, which together minimize the asymptotic variance of the estimator, are characterized and straightforward to estimate. When estimated weights and augmentation functions are substituted, the resulting treatment effect estimator remains consistent and asymptotically normal, and is asymptotically equivalent to the one based on the limiting weights and augmentation functions.

2 Preliminaries

We start by formulating the estimation problem using potential outcomes. For a generic patient in the study population, let $Y(a)$ denote the potential outcome for treatment $a$, where $a=1$ for an experimental treatment and $a=0$ for a control treatment, which may be placebo or standard of care. For each $a\in\{0,1\}$, we assume $\operatorname{E}\{Y(a)^{2}\}<\infty$ and write $\mu_{a}=\operatorname{E}\{Y(a)\}$. Suppose the treatment effect of interest is $\delta=g(\mu_{1})-g(\mu_{0})$, where $g$ is a smooth and strictly increasing function specified by the investigator. Popular choices for $g$ include identity (for continuous and other types of outcomes), logarithm (for outcomes with positive means), and logit (for binary outcomes). Let $\boldsymbol{W}$ be a vector of available baseline covariates (i.e., pre-treatment patient characteristics) that may be associated with either or both of $Y(1)$ and $Y(0)$.

An RCT takes a random sample of patients from the study population, assigns a treatment $A\in\{0,1\}$ to each participant in a randomized fashion, and measures the resulting outcome $Y=Y(A)=AY(1)+(1-A)Y(0)$. In current RCT practice, treatment assignment usually follows CIR, where $A$ is independent of all baseline variables. In particular, we have

[TABLE]

where $\pi$ is a pre-specified constant. In the CDR design proposed by Zhang et al. (2023), $A$ is allowed to depend on observed baseline covariates but not on potential outcomes. Specifically,

[TABLE]

where $p$ is a pre-specified propensity score function. The motivation for considering CDR, a generalization of CIR, is to achieve higher design efficiency (Zhang et al., 2023). A CDR design is somewhat similar to an observational study with the important distinction that the propensity score is specified and known by the investigator. In both cases (CIR and CDR), treatment assignment is independent across patients and follows the same mechanism for all patients. The resulting data can be conceptualized as independent copies of $(\boldsymbol{W},A,Y)$.

Under CIR, efficient estimation of $\delta$ entails making effective use of the information in $\boldsymbol{W}$, for which many methods are available; see Section 1 for an incomplete list of references. Under CDR, $\delta$ can be estimated using existing methods in the causal inference literature (e.g., van der Laan and Robins, 2003; van der Laan and Rose, 2011) coupled with the extra knowledge of the true propensity score; see Zhang et al. (2023, Section 3.1) for a detailed description. For both CIR and CDR, it makes sense to optimize treatment assignment by maximizing the nonparametric information bound. The optimal value of $\pi$ in CIR is

[TABLE]

where $g^{\prime}$ is the derivative function of $g$ and $v_{a}(\boldsymbol{W})=\operatorname{var}\{Y(a)|\boldsymbol{W}\}$, $a=0,1$. The optimal propensity score for CDR is given by

[TABLE]

Note that, in both cases, the optimal treatment assignment mechanism depends on the conditional variance functions $(v_{1},v_{0})$.

To take advantage of these optimality results in designing a trial, one may estimate $(v_{1},v_{0})$ from preliminary data and substitute the estimates into equation (1) or (2). This approach is infeasible if preliminary data are unavailable. Even with preliminary data available, this approach may be unreliable due to high variability and/or potential bias in the preliminary estimates of $(v_{1},v_{0})$. Indeed, early phase trials tend to be small, and the resulting estimates often lack precision. Sometimes preliminary data come from a confirmatory trial of a similar treatment, in which case one may be more concerned about potential bias (due to treatment differences) than about variability. These difficulties make it challenging to construct a reliable near-optimal design before the trial begins, and motivate our consideration of an adaptive design where we learn from accruing data and refine the treatment assignment mechanism as the trial proceeds.

3 Adaptive Design

For ease of presentation, we will focus on a simple two-stage CIR version of the proposed adaptive design in this section and the next one. The design and estimation method are extended to CDR in Appendix A and to multiple stages in Appendix B. The only adaptation we consider here is a possible change in the treatment allocation ratio following the interim analysis; we do not consider other adaptations (e.g., sample size, patient population, or outcome variable) or the possibility of early stopping for success or futility.

Let $(n_{1},n_{2})$ be pre-specified sample sizes for the two stages. Let $(\boldsymbol{W}_{si},Y_{si}(1),Y_{si}(0))$ denote baseline characteristics for patient $i=1,\dots,n_{s}$ in stage $s=1,2$; these are assumed to be independent across $(s,i)$ and identically distributed as $(\boldsymbol{W},Y(1),Y(0))$. Let $A_{si}$ denote the assigned treatment and $Y_{si}=Y_{si}(A_{si})=A_{si}Y_{si}(1)+(1-A_{si})Y_{si}(0)$ the resulting outcome for patient $i$ in stage $s$. The first stage follows a standard CIR design with a pre-specified probability $\pi_{1}\in(0,1)$ of assigning treatment 1. The resulting data, $\boldsymbol{D}_{1}=\{\boldsymbol{O}_{1i}=(\boldsymbol{W}_{1i},A_{1i},Y_{1i}),i=1,\dots,n_{1}\}$, are independent across patients and identically distributed as $\boldsymbol{O}_{1}=(\boldsymbol{W},A_{1},Y_{1})$, where $\operatorname{P}\{A_{1}=1|\boldsymbol{W},Y(1),Y(0)\}=\operatorname{P}(A_{1}=1)=\pi_{1}$ and $Y_{1}=Y(A_{1})=A_{1}Y(1)+(1-A_{1})Y(0)$.

At the interim analysis, $\boldsymbol{D}_{1}$ will be analyzed to estimate the marginal means $(\mu_{1},\mu_{0})$, the conditional variance functions $(v_{1},v_{0})$, and the optimal allocation $\pi_{\text{opt}}$. Estimating $(\mu_{1},\mu_{0})$ from $\boldsymbol{D}_{1}$ is a standard estimation problem for which many methods are available, including augmentation (Tsiatis et al., 2008). To estimate $(v_{1},v_{0})$ from $\boldsymbol{D}_{1}$, we note that $v_{a}(\boldsymbol{W})=\operatorname{var}(Y_{1}|A_{1}=a,\boldsymbol{W})$. If $\boldsymbol{W}$ is discrete, $v_{a}(\boldsymbol{w})$ can be estimated using the sample variance of $\{Y_{1i}:A_{1i}=a,\boldsymbol{W}_{1i}=\boldsymbol{w}\}$ for each $a\in\{0,1\}$ and each possible value $\boldsymbol{w}$ of $\boldsymbol{W}$. If $\boldsymbol{W}$ includes continuous components, estimation of $(v_{1},v_{0})$ may involve some modeling. For a binary outcome, a logistic regression model for $\operatorname{P}(Y_{1}=1|A_{1},\boldsymbol{W})$ would suffice for estimating $(v_{1},v_{0})$. For other types of outcomes, one could specify models for both $\operatorname{E}(Y_{1}|A_{1},\boldsymbol{W})$ and $\operatorname{var}(Y_{1}|A_{1},\boldsymbol{W})$ and estimate both models using estimating equations (e.g., Davidian and Carroll, 1987). Once $(\mu_{1},\mu_{0})$ and $(v_{1},v_{0})$ are estimated, their estimates can be substituted into equation (1) to estimate $\pi_{\text{opt}}$. The resulting estimate of $\pi_{\text{opt}}$, which we denote by $\pi_{2}$, will be used for treatment assignment in stage 2. If $n_{1}$ is insufficient for reliable estimation of $(v_{1},v_{0})$ based on $\boldsymbol{D}_{1}$, as may be the case if $\boldsymbol{W}$ has many components, the optimization process for choosing $\pi_{2}$ may be restricted to a coarsened version of $\boldsymbol{W}$, say $\boldsymbol{X}$, which may be a sub-vector or a discrete version of $\boldsymbol{W}$. Operationally, this means replacing $\boldsymbol{W}$ with $\boldsymbol{X}$ in this paragraph as well as equation (1); see Zhang et al. (2023, Section 5) for examples. Whether $\pi_{2}$ is optimized for all of $\boldsymbol{W}$ or a coarsened version of it has no impact on the subsequent development, where $\pi_{2}$ is allowed to depend on $\boldsymbol{D}_{1}$ in an arbitrary, unspecified fashion (subject to regularity conditions).

Once $\pi_{2}$ is chosen, the trial enters stage 2, which follows a standard CIR design where patients receive treatment 1 with probability $\pi_{2}$. Given $\pi_{2}$, the stage 2 data $\boldsymbol{D}_{2}=\{\boldsymbol{O}_{2i}=(\boldsymbol{W}_{2i},A_{2i},Y_{2i}),i=1,\dots,n_{2}\}$ are independent and identically distributed as $\boldsymbol{O}_{2}=(\boldsymbol{W},A_{2},Y_{2})$, where $\operatorname{P}\{A_{2}=1|\boldsymbol{W},Y(1),Y(0)\}=\operatorname{P}(A_{2}=1)=\pi_{2}$ and $Y_{2}=Y(A_{2})=A_{2}Y(1)+(1-A_{2})Y(0)$. Without conditioning on $\pi_{2}$, the $\boldsymbol{O}_{2i}$’s are identically distributed but not independent of each other. Because $\pi_{2}$ may depend on $\boldsymbol{D}_{1}$, the $\boldsymbol{O}_{2i}$’s are generally dependent on $\boldsymbol{D}_{1}$ as well. Thus, in general, the combined trial data $(\boldsymbol{D}_{1},\boldsymbol{D}_{2})$ are neither independent nor identically distributed across patients, and standard methods for estimating $\delta$ are not applicable to the current setting.

4 Treatment Effect Estimation

For each $a\in\{0,1\}$, possible estimators of $\mu_{a}$ include the stage-specific treatment group averages:

[TABLE]

where $\widehat{\operatorname{E}}_{s}$ denotes sample average over $\boldsymbol{D}_{s}=(\boldsymbol{O}_{s1},\dots,\boldsymbol{O}_{sn_{s}})$ and $I(\cdot)$ is the indicator function. By conditioning on $(A_{s1},\dots,A_{sn_{s}})$, it is easy to see that each $\widehat{\mu}_{a}^{(s)}$ is unbiased for $\mu_{a}$. The asymptotic property of $\widehat{\mu}_{a}^{(1)}$ is well known, while that of $\widehat{\mu}_{a}^{(2)}$ is less transparent due to the randomness of $\pi_{2}$. The two stage-specific estimators can be combined into a weighted average: $\widehat{\mu}_{a}(\theta)=\theta\widehat{\mu}_{a}^{(1)}+(1-\theta)\widehat{\mu}_{a}^{(2)}$, $\theta\in[0,1]$. Clearly, for any fixed $\theta$, $\widehat{\mu}_{a}(\theta)$ remains unbiased for $\mu_{a}$. The corresponding estimator of $\delta$ is $\widehat{\delta}(\theta)=g\{\widehat{\mu}_{1}(\theta)\}-g\{\widehat{\mu}_{0}(\theta)\}$. This estimator is generally inefficient as it does not make use of the available baseline covariate data. To incorporate baseline covariate data for improved efficiency, we consider a class of augmented estimators:

[TABLE]

where $b_{1}$ and $b_{2}$ are real-valued functions of $\boldsymbol{W}$ such that $\operatorname{E}\{b_{s}(\boldsymbol{W})^{2}\}<\infty$, $s=1,2$. The following theorem provides the asymptotic distribution of $\widehat{\delta}^{\text{aug}}(b_{1},b_{2},\theta)$ with fixed $(b_{1},b_{2},\theta)$.

Theorem 1**.**

Under the two-stage adaptive CIR design described in Section 3, assume that, as $n_{1}\rightarrow\infty$, $n_{1}/n_{2}$ converges to some $\lambda\in(0,\infty)$ and $\pi_{2}$ converges in probability to some $\pi_{2}^{*}\in(0,1)$. Then, for any fixed $(b_{1},b_{2},\theta)$, we have, as $n_{1}\rightarrow\infty$,

[TABLE]

where

[TABLE]

and $\boldsymbol{O}_{2}^{*}=(\boldsymbol{W},A_{2}^{*},Y_{2}^{*})$ is similar to $\boldsymbol{O}_{2}$ with $\pi_{2}^{*}$ replacing $\pi_{2}$:

[TABLE]

The assumption that $\pi_{2}$ converges to some $\pi_{2}^{*}$ is expected to hold if the underlying estimates of $(v_{1},v_{0})$ (described in Section 3) converge to well-defined limits.

It is of interest to minimize $\sigma_{\textsc{cir}}^{2}(b_{1},b_{2},\theta)$ with respect to $(b_{1},b_{2},\theta)$. For any given $\theta$, $\sigma_{\textsc{cir}}^{2}(b_{1},b_{2},\theta)$ is minimized by setting $(b_{1},b_{2})$ equal to

[TABLE]

where $m_{a}(\boldsymbol{W})=\operatorname{E}\{Y(a)|\boldsymbol{W}\}$, $a=0,1$. A proof of this result is given in Appendix C. The optimal choices $(b_{1,\text{opt}},b_{2,\text{opt}})$ are generally unknown in reality and need to be estimated from trial data. Suppose $\widetilde{b}_{s}$ is an estimator of $b_{s,\text{opt}}$ that converges in probability to some limit function $b_{s}$ in the sense that $\|\widetilde{b}_{s}-b_{s}\|_{2}=o_{p}(1)$, where $s=1,2$ and $\|h\|_{2}=[\operatorname{E}\{h(\boldsymbol{W})^{2}\}]^{1/2}$. Under regularity conditions, $\widehat{\delta}^{\text{aug}}(\widetilde{b}_{1},\widetilde{b}_{2},\theta)$ is asymptotically normal with asymptotic variance $\sigma_{\textsc{cir}}^{2}(b_{1},b_{2},\theta)$ and thus asymptotically equivalent to $\widehat{\delta}^{\text{aug}}(b_{1},b_{2},\theta)$ (see Theorem 2 below).

Motivated by expression (3), we consider

[TABLE]

where $(\widehat{\mu}_{1},\widehat{\mu}_{0})$ and $(\widehat{m}_{1},\widehat{m}_{0})$ are generic estimators of $(\mu_{1},\mu_{0})$ and $(m_{1},m_{0})$. Relevant to the estimation of $(m_{1},m_{0})$ is the fact that $m_{a}(\boldsymbol{W})=\operatorname{E}(Y_{1}|A_{1}=a,\boldsymbol{W})=\operatorname{E}(Y_{2}|A_{2}=a,\boldsymbol{W})$, $a=0,1$; this holds despite $\boldsymbol{O}_{1}$ and $\boldsymbol{O}_{2}$ not being identically distributed. If $\boldsymbol{W}$ is discrete, $m_{a}(\boldsymbol{w})$ can be estimated by the sample mean of $\{Y_{i}:A_{i}=a,\boldsymbol{W}_{i}=\boldsymbol{w}\}$ for each $a\in\{0,1\}$ and each possible value $\boldsymbol{w}$ of $\boldsymbol{W}$. More generally, $(m_{1},m_{0})$ may be estimated by fitting an outcome regression model to the totality of trial data. We assume that $(\widehat{\mu}_{1},\widehat{\mu}_{0})$ converge to $(\mu_{1},\mu_{0})$ and $(\widehat{m}_{1},\widehat{m}_{0})$ to some limit functions $(m_{1}^{*},m_{0}^{*})$; see Appendix C for a discussion of this assumption. It follows that $(\widehat{b}_{1},\widehat{b}_{2})$ converge to $(b_{1}^{*},b_{2}^{*})$ defined by

[TABLE]

For this particular construction, we show in Appendix C that $\sigma_{\textsc{cir}}^{2}(b_{1}^{*}(\cdot;\theta),b_{2}^{*}(\cdot;\theta),\theta)$, the asymptotic variance of $\widehat{\delta}^{\text{aug}}(\widehat{b}_{1}(\cdot;\theta),\widehat{b}_{2}(\cdot;\theta),\theta)$, is minimized by setting $\theta$ equal to $\theta_{\text{opt}}^{*}=\lambda\sigma_{2,\textsc{cir}}^{*2}/(\sigma_{1,\textsc{cir}}^{*2}+\lambda\sigma_{2,\textsc{cir}}^{*2})$, where

[TABLE]

Remark 1*.*

If $(m_{1}^{*},m_{0}^{*})=(m_{1},m_{0})$, then $\sigma_{1,\textsc{cir}}^{*2}$ is the asymptotic variance of a nonparametric efficient estimator of $\delta$ based on stage 1 data, and $\sigma_{2,\textsc{cir}}^{*2}$ has a similar interpretation pertaining to stage 2 with the random variable $\pi_{2}$ replaced by its limit $\pi_{2}^{*}$. It is easy to see that, with $(m_{1}^{*},m_{0}^{*})=(m_{1},m_{0})$, the triplet $(b_{1}^{*}(\cdot;\theta_{\text{opt}}^{*}),b_{2}^{*}(\cdot;\theta_{\text{opt}}^{*}),\theta_{\text{opt}}^{*})$ is the (essentially) unique minimizer of $\sigma_{\textsc{cir}}^{2}(b_{1},b_{2},\theta)$. The global minimum of $\sigma_{\textsc{cir}}^{2}(b_{1},b_{2},\theta)$, as a function of the design parameters $(\pi_{1},\pi_{2}^{*})$, is minimized when $\pi_{1}=\pi_{2}^{*}=\pi_{\text{opt}}$; this follows from the same argument for proving Theorem 1 of Zhang et al. (2023). Heuristically, the efficiency of the adaptive design in Section 3 depends on how close $\pi_{1}$ and $\pi_{2}^{*}$ are to the optimal value $\pi_{\text{opt}}$. Compared to a one-stage non-adaptive design that randomizes $n_{1}+n_{2}$ patients according to $\pi_{1}$, the proposed adaptive design is more efficient if $\pi_{2}^{*}$ is more efficient than $\pi_{1}$ in the sense that $\sigma_{2,\textsc{cir}}^{*2}<\sigma_{1,\textsc{cir}}^{*2}$ when $(m_{1}^{*},m_{0}^{*})=(m_{1},m_{0})$.

For any $(m_{1}^{*},m_{0}^{*})$, $\theta_{\text{opt}}^{*}$ is estimated consistently by $\widehat{\theta}=n_{1}\widehat{\sigma}_{2,\textsc{cir}}^{2}/(n_{2}\widehat{\sigma}_{1,\textsc{cir}}^{2}+n_{1}\widehat{\sigma}_{2,\textsc{cir}}^{2})$ with

[TABLE]

where $\widehat{\operatorname{var}}_{s}$ denotes sample variance over $\boldsymbol{D}_{s}=(\boldsymbol{O}_{s1},\dots,\boldsymbol{O}_{sn_{s}})$, $s=1,2$. The consistency of $\widehat{\theta}$ is proved in Appendix C. It is natural to substitute $\widehat{\theta}$ into $\widehat{\delta}^{\text{aug}}(\widehat{b}_{1}(\cdot;\theta),\widehat{b}_{2}(\cdot;\theta),\theta)$ and estimate $\delta$ using $\widehat{\delta}^{\text{aug}}(\widehat{b}_{1}(\cdot;\widehat{\theta}),\widehat{b}_{2}(\cdot;\widehat{\theta}),\widehat{\theta})$, whose asymptotic property is given in the next theorem. For brevity, we write $\widehat{b}_{s}=\widehat{b}_{s}(\cdot;\widehat{\theta})$ and $b_{s}^{*}=b_{s}^{*}(\cdot;\theta_{\text{opt}}^{*})$, $s=1,2$.

Theorem 2**.**

Assume the conditions in Theorem 1 hold. For any random triplet $(\widetilde{b}_{1},\widetilde{b}_{2},\widetilde{\theta})$ that element-wise converges in probability to a fixed triplet $(b_{1},b_{2},\theta)$, we have, as $n_{1}\rightarrow\infty$,

[TABLE]

that is, $\widehat{\delta}^{\text{aug}}(\widetilde{b}_{1},\widetilde{b}_{2},\widetilde{\theta})$ is asymptotically equivalent to $\widehat{\delta}^{\text{aug}}(b_{1},b_{2},\theta)$. In particular, for $(\widehat{b}_{1},\widehat{b}_{2},\widehat{\theta})$ defined in the preceding paragraph, $\widehat{\delta}^{\text{aug}}(\widehat{b}_{1},\widehat{b}_{2},\widehat{\theta})$ is asymptotically equivalent to $\widehat{\delta}^{\text{aug}}(b_{1}^{*},b_{2}^{*},\theta_{\text{opt}}^{*})$ and attains the smallest asymptotic variance among $\{\widehat{\delta}^{\text{aug}}(\widehat{b}_{1}(\cdot;\theta),\widehat{b}_{2}(\cdot;\theta),\theta),\theta\in[0,1]\}$. If $(m_{1}^{*},m_{0}^{*})=(m_{1},m_{0})$, then $\widehat{\delta}^{\text{aug}}(\widehat{b}_{1},\widehat{b}_{2},\widehat{\theta})$ attains the smallest asymptotic variance among all estimators of the form $\widehat{\delta}^{\text{aug}}(b_{1},b_{2},\theta)$.

The asymptotic variance of $\widehat{\delta}^{\text{aug}}(\widehat{b}_{1},\widehat{b}_{2},\widehat{\theta})$, $\sigma_{\textsc{cir}}^{2}(b_{1}^{*},b_{2}^{*},\theta_{\text{opt}}^{*})$, is consistently estimated by

[TABLE]

5 Simulation

This section reports a simulation study evaluating the two-stage adaptive CIR and CDR designs together with the proposed estimation methods. In this simulation study, the covariate vector $\boldsymbol{W}=(W_{1},W_{2},W_{3})^{\prime}$ follows a trivariate normal distribution with mean ${\bf 0}$ and a variance matrix whose diagonal elements are all equal to 0.5 and off-diagonal elements equal to 0. We consider a binary outcome which, for each $a\in\{0,1\}$, relates to $\boldsymbol{W}$ through a logistic regression model: \operatorname{logit}[\operatorname{P}\{Y(a)=1|\boldsymbol{W}\}]=\gamma_{0}+\gamma_{1}a+\mbox{\boldmath{\gamma}}_{2}^{\prime}\boldsymbol{W}+\mbox{\boldmath{\gamma}}_{3}^{\prime}(a\boldsymbol{W}), where $\gamma_{0}=-2.5$, $\gamma_{1}=1$ or 2, \mbox{\boldmath{\gamma}}_{2}=(-0.2,-0.2,0.2)^{\prime}, and \mbox{\boldmath{\gamma}}_{3}=(1,-1,-1.5)^{\prime}. The treatment effect of interest is the log-odds ratio $\delta=\operatorname{logit}[\operatorname{P}\{Y(1)=1\}]-\operatorname{logit}[\operatorname{P}\{Y(0)=1\}].$

Under a two-stage design, treatment assignment follows a pre-specified mechanism in stage 1 and an optimized one (based on all available data at the interim analysis) in stage 2. The treatment assignment mechanism for stage 1 may or may not be informed by preliminary data, giving rise to two distinct settings of practical interest:

Setting 1: Stage 1 follows the standard 1:1 CIR design with $\pi_{1}=1/2$;

Setting 2: Stage 1 follows a CIR or CDR design optimized using preliminary data.

In each setting, for each two-stage design we consider, a one-stage design that utilizes the same stage 1 treatment assignment mechanism throughout the trial is included as a comparator. We set $n_{1}=n_{2}=250$ for all two-stage designs and $n=500$ (total sample size) for one-stage designs. Each design is simulated 5,000 times in each distinct situation.

5.1 Setting 1

In Setting 1, we compare three designs: a one-stage 1:1 CIR design, a two-stage CIR design with 1:1 CIR (stage 1) followed by optimized CIR (stage 2), and a two-stage CDR design with 1:1 CIR followed by optimized CDR. In the last two designs, treatment allocation in stage 2 is optimized for $\boldsymbol{X}$, a non-empty sub-vector of $\boldsymbol{W}$, using stage 1 data. For each choice of $\boldsymbol{X}$, the optimal treatment allocation ($\pi_{\text{opt}}$ or $p_{\text{opt}}$) is estimated by substituting estimates of $(\mu_{1},\mu_{0})$ and $(v_{1},v_{0})$ from stage 1 data into equation (1) or (2). Stage 1 estimates of $(v_{1},v_{0})$ are obtained by fitting the following logistic regression model:

[TABLE]

This working model is misspecified unless $\boldsymbol{X}=\boldsymbol{W}$.

Under each design, $\delta$ is estimated using two estimators: a simple one and an optimized one. For the two-stage CIR design, the simple estimator is $\widehat{\delta}(\theta_{0})$, where $\theta_{0}=n_{1}/(n_{1}+n_{2})=1/2$, and the optimized estimator is $\widehat{\delta}^{\text{aug}}(\widehat{b}_{1},\widehat{b}_{2},\widehat{\theta})$. As described in Section 4, $(\widehat{b}_{1},\widehat{b}_{2})$ are based on $(\widehat{m}_{1},\widehat{m}_{0})$, which in this case are obtained by fitting a logistic regression model similar to (4), with $\boldsymbol{X}$ replaced by $\boldsymbol{W}$, to all trial data (from both stages). For the two-stage CDR design, the simple estimator is $\widehat{\delta}^{\text{ipw}}(\eta_{0})$, where $\eta_{0}=\theta_{0}$, and the optimized estimator is $\widehat{\delta}^{\text{aipw}}(\widehat{c}_{1},\widehat{c}_{2},\widehat{\eta})$, where $(\widehat{c}_{1},\widehat{c}_{2})$ are based on the same $(\widehat{m}_{1},\widehat{m}_{0})$ mentioned earlier (both estimators are described in Appendix A). For the one-stage design, the simple estimator is based on treatment-specific averages and the optimized one is an augmented estimator described in Zhang et al. (2023, Section 2).

As expected, estimation bias is negligible for all estimators and all designs (results not shown). Table 1 reports relative efficiency results with the simple estimator under the one-stage design serving as the reference. The relative efficiency of an estimator is calculated as the inverse ratio of its empirical variance to that of the reference estimator. Under each design, the optimized estimator is clearly advantageous to the simple estimator. Comparing the optimized estimators for the three designs, the two-stage CIR design is consistently more efficient than the one-stage design, and the two-stage CDR design is even more efficient. Apparently, the stage 1 data with $n_{1}=250$ provide adequate support for estimating the optimal CDR design. For the optimized estimators under the two-stage designs, Table S1 in Appendix C indicates that the proposed variance estimators are virtually median-unbiased, and Table 2 demonstrates that the resulting confidence intervals provide close-to-nominal coverage.

5.2 Setting 2

In Setting 2, we compare five different designs: a one-stage CIR design with optimized CIR, a one-stage CDR design with optimized CDR, a two-stage CIR design with optimized CIR (stage 1) followed by further optimized CIR (stage 2), a two-stage CDR design with optimized CDR followed by further optimized CDR, and a two-stage hybrid design with optimized CIR followed by optimized CDR. All attempts to optimize treatment allocation make use of a preliminary dataset of size $n_{0}=100$, generated under 1:1 CIR in the same patient population described earlier. (The optimizations for stage 2 also make use of stage 1 data.) As in Setting 1, each optimization is based on a sub-vector $\boldsymbol{X}$ of $\boldsymbol{W}$ and involves fitting the logistic regression model (4) to the available data.

Under each design, $\delta$ is estimated using a simple estimator and an optimized one. For the one-stage CIR design and the two-stage designs, these are the same estimators described and compared in Section 5.1. (The two-stage hybrid design is approached as a two-stage CDR design for the purpose of treatment effect estimation.) For the one-stage CDR design, the simple estimator is an IPW estimator and the optimized estimator is an AIPW estimator; both are described in Zhang et al. (2023, Section 3.1).

The relative efficiency results, with the optimized estimator under the one-stage CIR design as the reference, are shown in the lower section of Table 1. The one-stage CDR design generally underperforms the one-stage CIR design, suggesting that the preliminary data with $n_{0}=100$ may be insufficient to support full optimization of CDR. The two-stage CIR and CDR designs perform similarly to each other, and both designs generally outperform the one-stage designs. The two-stage hybrid design appears to perform even better than the two-stage CIR and CDR designs, though the incremental improvement tends to be small. To understand the superior performance of the two-stage hybrid design, we note that its first stage (optimized CIR) acknowledges the limited amount ($n_{0}=100$) of preliminary data and its second stage (optimized CDR) takes full advantage of the combined data ($n_{0}+n_{1}=350$) available at the interim. Tables 2 and S1 demonstrate adequate coverage and variance estimation for the two-stage optimized estimators.

5.3 Additional Simulation Results

Appendix D reports the results of additional simulation studies with various deviations from the main study described here: a smaller sample size ($n=300$), different sample size allocations between stages, inclusion of non-prognostic covariates, and more severe misspecification of the working model for interim estimation of $(v_{1},v_{0})$. The results are generally consistent with those reported here, with a few new insights (detailed in Appendix D).

6 Application

As an illustration, this section describes how the proposed methodology can be used to re-design and re-analyze the NINDS rt-PA trial introduced in Section 1. Following previous analyses of the same trial (NINDS rt-PA Stroke Study Group, 1995; Ingall et al., 2004; Zhang and Ma, 2019), we will compare treatments at 3 months post-randomization with respect to four clinical outcomes: Barthel Index, Modified Rankin Scale, Glasgow Outcome Scale, and National Institutes of Health Stroke Scale (NIHSS). For ease of interpretation, each outcome is linearly re-scaled to the unit interval in such a way that higher values are desirable. For each outcome, the efficacy of rt-PA relative to placebo is measured by the between-group difference (rt-PA minus placebo) in the re-scaled mean outcome (i.e., $\delta=\mu_{1}-\mu_{0}$).

Our objective is to improve treatment assignment for statistical efficiency by incorporating baseline covariates. For this purpose, we consider three baseline covariates: age, baseline NIHSS, and history of diabetes. History of diabetes is inherently dichotomous, and the other two covariates are dichotomized at their respective medians. The three dichotomous covariates together comprise the covariate vector $\boldsymbol{W}$. As alternatives to the original 1:1 CIR design, we consider a two-stage CIR design (1:1 CIR followed by optimized CIR) and a two-stage CDR design (1:1 CIR followed by optimized CDR). Without actual enrollment information, we randomly partition the trial participants into two stages in a 1:1 ratio. We use stage 1 data to optimize treatment allocation in stage 2 based on a covariate vector $\boldsymbol{X}$, which consists of one or more of the three dichotomous covariates described earlier. The conditional variance functions $(v_{1},v_{0})$ are estimated empirically, as indicated in Section 3. For each patient in stage 2, $A_{2i}$ is assigned randomly according to the optimized CIR or CDR design, and $Y_{2i}$ is chosen randomly from the observed outcome values among the actual trial participants in the same treatment group with the same covariate value. Only one set of trial data is generated for each design. We estimate $\delta$ using optimized estimators that are virtually identical to those described in Section 5.1 with two minor differences. First, the estimand here is a mean difference rather than a log-odds ratio. Second, the discrete nature of the current $\boldsymbol{W}$ allows $(m_{1},m_{0})$ to be estimated empirically using sample averages.

Table 3 reports the results of this analysis (point estimates of $\delta$, standard errors, and relative efficiency results) for the four clinical outcomes and different choices of $\boldsymbol{X}$. The relative efficiency is calculated as a ratio of variance estimates, with the one-stage CIR design as the reference. As expected, standard errors are generally smaller for the two-stage designs as compared to the one-stage design. The reductions in standard errors may be small on the absolute scale but can be substantial on the relative efficiency scale. Point estimates are typically larger under the two-stage designs as compared to the one-stage design. Taken together, the results in Table 3 clearly indicate that the use of rt-PA has a beneficial effect, although its statistical significance is difficult to quantify due to multiplicity issues.

7 Discussion

An important practical question in applying the proposed design is how to choose stage-specific sample sizes. For example, in a two-stage design with a fixed total sample size, increasing $n_{1}$ improves estimation precision in the interim analysis but reduces the impact of the estimated optimal design (due to decreased $n_{2}$). We have investigated this trade-off in a simulation study reported in Appendix D, and the results suggest that the optimal allocation between stages might be some intermediate value. Further research is needed to better characterize that optimal allocation and provide helpful guidance to practitioners.

In this article, we have focused on treatment assignment as the only adaptable aspect of the trial, without considering other possible adaptations (e.g., sample size re-estimation). It is of interest to consider how to make the design more flexible by incorporating other possible adaptations while maintaining the integrity of the trial.

Acknowledgements

The research of Wei Zhang was supported by the National Key R&D Program of China (2022YFA1004800). The research of Aiyi Liu was supported by the intramural research program of the Eunice Kennedy Shriver National Institute of Child Health and Human Development.

Supporting Information

Web Appendices and Tables referenced in Sections 3-5, and data and code referenced in Section 5 are available with this paper at the Biometrics website on Oxford Academic.

Data Availability Statement

The NINDS rt-PA trial data are available from the corresponding author upon reasonable request.

Appendix A: Extension to CDR

In this appendix, we extend the adaptive design in Section 3 and the estimation method in Section 4 to the CDR setting. We take Sections 3 and 4 as background information and focus attention on important differences due to CDR. Unless otherwise stated, the notations in Sections 3 and 4 remain applicable in the present setting.

Let $p_{1}(\boldsymbol{W})$ be the pre-specified propensity score for treatment assignment in stage 1. The stage 1 data, $\boldsymbol{D}_{1}=\{\boldsymbol{O}_{1i}=(\boldsymbol{W}_{1i},A_{1i},Y_{1i}),i=1,\dots,n_{1}\}$, are independent across patients and identically distributed as $\boldsymbol{O}_{1}=(\boldsymbol{W},A_{1},Y_{1})$, where $\operatorname{P}\{A_{1}=1|\boldsymbol{W},Y(1),Y(0)\}=\operatorname{P}(A_{1}=1|\boldsymbol{W})=p_{1}(\boldsymbol{W})$ and $Y_{1}=Y(A_{1})=A_{1}Y(1)+(1-A_{1})Y(0)$. At the interim analysis, $\boldsymbol{D}_{1}$ will be used to estimate $(\mu_{1},\mu_{0})$, $(v_{1},v_{0})$, and the optimal propensity score $p_{\text{opt}}$. Estimation of $(\mu_{1},\mu_{0})$ in a single-stage CDR trial is discussed in Zhang et al. (2023, Section 3.1). Estimation of $(v_{1},v_{0})$ follows the same considerations given in Section 3 because CDR implies $v_{a}(\boldsymbol{W})=\operatorname{var}(Y_{1}|A_{1}=a,\boldsymbol{W})$, $a=0,1$. Estimates of $(\mu_{1},\mu_{0})$ and $(v_{1},v_{0})$ can be substituted into equation (2) to obtain an estimate of $p_{\text{opt}}$, say $p_{2}$, which will then be used as the propensity score function in stage 2. Given $p_{2}$, the stage 2 data $\boldsymbol{D}_{2}=\{\boldsymbol{O}_{2i}=(\boldsymbol{W}_{2i},A_{2i},Y_{2i}),i=1,\dots,n_{2}\}$ are independent and identically distributed as $\boldsymbol{O}_{2}=(\boldsymbol{W},A_{2},Y_{2})$, where $\operatorname{P}\{A_{2}=1|\boldsymbol{W},Y(1),Y(0)\}=\operatorname{P}(A_{2}=1|\boldsymbol{W})=p_{2}(\boldsymbol{W})$ and $Y_{2}=Y(A_{2})=A_{2}Y(1)+(1-A_{2})Y(0)$. As noted in Section 3, the combined trial data $(\boldsymbol{D}_{1},\boldsymbol{D}_{2})$ are neither independent nor identically distributed across patients.

The rest of this section is focused on treatment effect estimation. To account for CDR, we replace the initial stage-specific estimators $\widehat{\mu}_{a}^{(s)}$ with inverse probability weighted (IPW) estimators:

[TABLE]

These can be combined as in Section 4 to yield $\widehat{\mu}_{a}^{\text{ipw}}(\eta)=\eta\widehat{\mu}_{a}^{\text{ipw}(1)}+(1-\eta)\widehat{\mu}_{a}^{\text{ipw}(2)}$, $a=0,1$, and $\widehat{\delta}^{\text{ipw}}(\eta)=g\{\widehat{\mu}_{1}^{\text{ipw}}(\eta)\}-g\{\widehat{\mu}_{0}^{\text{ipw}}(\eta)\}$, $\eta\in[0,1]$. For improved efficiency, we consider a class of augmented IPW (AIPW) estimators of $\delta$:

[TABLE]

where $c_{1}$ and $c_{2}$ are real-valued functions of $\boldsymbol{W}$ such that $\operatorname{E}\{c_{s}(\boldsymbol{W})^{2}\}<\infty$, $s=1,2$. The asymptotic distribution of $\widehat{\delta}^{\text{aipw}}(c_{1},c_{2},\eta)$ with fixed $(c_{1},c_{2},\eta)$ is given in the next theorem, which is analogous to Theorem 1.

Theorem S1**.**

Under the two-stage adaptive CDR design described in this section, assume that, as $n_{1}\rightarrow\infty$, $n_{1}/n_{2}$ converges to some $\lambda\in(0,\infty)$ and $\|p_{2}-p_{2}^{*}\|_{2}=o_{p}(1)$ for some real-valued function $p_{2}^{*}$ such that $\operatorname{E}\{p_{2}^{*}(\boldsymbol{W})^{2}\}<\infty$. Then, for any fixed $(c_{1},c_{2},\eta)$, we have, as $n_{1}\rightarrow\infty$,

[TABLE]

where

[TABLE]

and $\boldsymbol{O}_{2}^{*}=(\boldsymbol{W},A_{2}^{*},Y_{2}^{*})$ is similar to $\boldsymbol{O}_{2}$ with $p_{2}^{*}$ replacing $p_{2}$:

[TABLE]

The asymptotic variance $\sigma_{\textsc{cdr}}^{2}(c_{1},c_{2},\eta)$ can be minimized in two steps, in a similar fashion to the minimization of $\sigma_{\textsc{cir}}^{2}(b_{1},b_{2},\theta)$ in Section 4. We show in Appendix C that, for any given $\eta$, $\sigma_{\textsc{cdr}}^{2}(c_{1},c_{2},\eta)$ is minimized by setting $(c_{1},c_{2})$ equal to

[TABLE]

where $m_{a}(\boldsymbol{W})=\operatorname{E}\{Y(a)|\boldsymbol{W}\}=\operatorname{E}(Y_{s}|A_{s}=a,\boldsymbol{W})$, $a=0,1$, $s=1,2$. These optimal choices are estimated by

[TABLE]

where $(\widehat{\mu}_{1},\widehat{\mu}_{0})$ and $(\widehat{m}_{1},\widehat{m}_{0})$ are generic estimators of $(\mu_{1},\mu_{0})$ and $(m_{1},m_{0})$. We assume that $(\widehat{\mu}_{1},\widehat{\mu}_{0})$ converge to $(\mu_{1},\mu_{0})$ and $(\widehat{m}_{1},\widehat{m}_{0})$ to some limit functions $(m_{1}^{*},m_{0}^{*})$; then $(\widehat{c}_{1},\widehat{c}_{2})$ converge to $(c_{1}^{*},c_{2}^{*})$ defined by

[TABLE]

We show in Appendix C that $\sigma_{\textsc{cdr}}^{2}(c_{1}^{*}(\cdot;\eta),c_{2}^{*}(\cdot;\eta),\eta)$ is minimized by setting $\eta$ equal to $\eta_{\text{opt}}^{*}=\lambda\sigma_{2,\textsc{cdr}}^{*2}/(\sigma_{1,\textsc{cdr}}^{*2}+\lambda\sigma_{2,\textsc{cdr}}^{*2})$, where

[TABLE]

Remark 1 is easily extensible to the present setting; for example, $\sigma_{\textsc{cdr}}^{2}(c_{1},c_{2},\eta)$ is minimized by the triplet $(c_{1}^{*}(\cdot;\eta_{\text{opt}}^{*}),c_{2}^{*}(\cdot;\eta_{\text{opt}}^{*}),\eta_{\text{opt}}^{*})$ with $(m_{1}^{*},m_{0}^{*})=(m_{1},m_{0})$, and the minimum value of $\sigma_{\textsc{cdr}}^{2}(c_{1},c_{2},\eta)$, as a function of the design parameters $(p_{1},p_{2}^{*})$, is minimized when $p_{1}=p_{2}^{*}=p_{\text{opt}}$. Regardless of $(m_{1}^{*},m_{0}^{*})$, $\eta_{\text{opt}}^{*}$ is estimated consistently by $\widehat{\eta}=n_{1}\widehat{\sigma}_{2,\textsc{cdr}}^{2}/(n_{2}\widehat{\sigma}_{1,\textsc{cdr}}^{2}+n_{1}\widehat{\sigma}_{2,\textsc{cdr}}^{2})$ with

[TABLE]

Based on these observations, we propose to estimate $\delta$ using $\widehat{\delta}^{\text{aipw}}(\widehat{c}_{1}(\cdot;\widehat{\eta}),\widehat{c}_{2}(\cdot;\widehat{\eta}),\widehat{\eta})$. Its asymptotic property is given in the next theorem, where we abbreviate $\widehat{c}_{s}=\widehat{c}_{s}(\cdot;\widehat{\eta})$ and $c_{s}^{*}=c_{s}^{*}(\cdot;\eta_{\text{opt}}^{*})$, $s=1,2$.

Theorem S2**.**

Assume the conditions in Theorem S1 hold. For any random triplet $(\widetilde{c}_{1},\widetilde{c}_{2},\widetilde{\eta})$ that element-wise converges in probability to a fixed triplet $(c_{1},c_{2},\eta)$, we have, as $n_{1}\rightarrow\infty$,

[TABLE]

that is, $\widehat{\delta}^{\text{aipw}}(\widetilde{c}_{1},\widetilde{c}_{2},\widetilde{\eta})$ is asymptotically equivalent to $\widehat{\delta}^{\text{aipw}}(c_{1},c_{2},\eta)$. In particular, for $(\widehat{c}_{1},\widehat{c}_{2},\widehat{\eta})$ defined in the preceding paragraph, $\widehat{\delta}^{\text{aipw}}(\widehat{c}_{1},\widehat{c}_{2},\widehat{\eta})$ is asymptotically equivalent to $\widehat{\delta}^{\text{aipw}}(c_{1}^{*},c_{2}^{*},\eta_{\text{opt}}^{*})$ and attains the smallest asymptotic variance among $\{\widehat{\delta}^{\text{aipw}}(\widehat{c}_{1}(\cdot;\eta),\widehat{c}_{2}(\cdot;\eta),\eta),\eta\in[0,1]\}$. If $(m_{1}^{*},m_{0}^{*})=(m_{1},m_{0})$, then $\widehat{\delta}^{\text{aipw}}(\widehat{c}_{1},\widehat{c}_{2},\widehat{\eta})$ attains the smallest asymptotic variance among all estimators of the form $\widehat{\delta}^{\text{aipw}}(c_{1},c_{2},\eta)$.

The asymptotic variance of $\widehat{\delta}^{\text{aipw}}(\widehat{c}_{1},\widehat{c}_{2},\widehat{\eta})$, $\sigma_{\textsc{cdr}}^{2}(c_{1}^{*},c_{2}^{*},\eta_{\text{opt}}^{*})$, is consistently estimated by

[TABLE]

Appendix B: Extension to Multiple Stages

We now extend the two-stage adaptive CDR design and estimation method in Appendix A to an arbitrary (specified) number of stages. Because CDR is strictly more general than CIR, the multi-stage CDR design described here can be reduced to a multi-stage CIR design by restricting the randomization mechanism to CIR at each stage, and the related estimation method remains applicable with propensity scores replaced by constants.

Let us consider an adaptive CDR trial with $k\geq 2$ stages and pre-specified sample sizes $n_{s}$, $s=1,\dots,k$. The trial data consist of $(\boldsymbol{D}_{1},\dots,\boldsymbol{D}_{k})$, where $\boldsymbol{D}_{s}$ denotes the patient-level data from stage $s$: $\{\boldsymbol{O}_{si}=(\boldsymbol{W}_{si},A_{si},Y_{si}),i=1,\dots,n_{s}\}$. The first two stages are exactly as described in Section 5. If $k>2$, a second interim analysis is conducted after stage 2 to re-estimate $(\mu_{1},\mu_{0})$, $(v_{1},v_{0})$ and $p_{\text{opt}}$ on the basis of $(\boldsymbol{D}_{1},\boldsymbol{D}_{2})$. The updated estimate of $p_{\text{opt}}$, say $p_{3}$, is then used as the propensity score function for treatment assignment in stage 3. This process is iterated until all $k$ stages are completed. For each $s\geq 2$, upon conditioning on $p_{s}$, the $\boldsymbol{O}_{si}$’s are independent across $i$ and identically distributed as $\boldsymbol{O}_{s}=(\boldsymbol{W},A_{s},Y_{s})$, where $\operatorname{P}\{A_{s}=1|\boldsymbol{W},Y(1),Y(0)\}=\operatorname{P}(A_{s}=1|\boldsymbol{W})=p_{s}(\boldsymbol{W})$ and $Y_{s}=Y(A_{s})=A_{s}Y(1)+(1-A_{s})Y(0)$.

The treatment effect estimation problem here resembles the one considered in Appendix A; therefore, we will omit theoretical details and proofs and focus on describing the estimation method and key results. For each $a\in\{0,1\}$, let $\widehat{\mu}_{a}^{\text{ipw}(s)}$ be defined as in Appendix A for $s=1,\dots,k$, and let \widehat{\mu}_{a}^{\text{ipw}}(\mbox{\boldmath{\eta}})=\sum_{s=1}^{k}\eta_{s}\widehat{\mu}_{a}^{\text{ipw}(s)}, where \mbox{\boldmath{\eta}}=(\eta_{1},\dots,\eta_{k}) is a collection of normalized weights satisfying $\eta_{s}\geq 0$ $\forall s$ and $\sum_{s=1}^{k}\eta_{s}=1$. A generic AIPW estimator of $\delta$ is given by

[TABLE]

where \widehat{\delta}^{\text{ipw}}(\mbox{\boldmath{\eta}})=g(\widehat{\mu}_{1}^{\text{ipw}}(\mbox{\boldmath{\eta}}))-g(\widehat{\mu}_{0}^{\text{ipw}}(\mbox{\boldmath{\eta}})) and $\boldsymbol{c}=(c_{1},\dots,c_{k})$ is a collection of real-valued functions of $\boldsymbol{W}$ such that $\operatorname{E}\{c_{s}(\boldsymbol{W})^{2}\}<\infty$, $s=1,\dots,k$. For each $s\in\{2,\dots,k\}$, we assume that, as $n_{1}\rightarrow\infty$, $n_{1}/n_{s}$ converges to some $\lambda_{s}\in(0,\infty)$ and $p_{s}$ converges to a limit function $p_{s}^{*}$ in the sense described in Theorem S1. (For notational convenience, we define $\lambda_{1}=1$ and $p_{1}^{*}=p_{1}$.) Then, for any fixed (\boldsymbol{c},\mbox{\boldmath{\eta}}), \sqrt{n_{1}}\{\widehat{\delta}^{\text{aipw}}(\boldsymbol{c},\mbox{\boldmath{\eta}})-\delta\} converges to a zero-mean normal distribution whose variance is denoted by \sigma_{\textsc{cdr}}^{2}(\boldsymbol{c},\mbox{\boldmath{\eta}}).

For any given ${\eta}$, the asymptotic variance \sigma_{\textsc{cdr}}^{2}(\boldsymbol{c},\mbox{\boldmath{\eta}}) is minimized by setting \boldsymbol{c}=\boldsymbol{c}_{\text{opt}}(\cdot;\mbox{\boldmath{\eta}}), where $\boldsymbol{c}_{\text{opt}}=(c_{1,\text{opt}},\dots,c_{k,\text{opt}})$ with

[TABLE]

The optimal choice $\boldsymbol{c}_{\text{opt}}$ is estimated by $\widehat{\boldsymbol{c}}=(\widehat{c}_{1},\dots,\widehat{c}_{k})$, where

[TABLE]

With $(\widehat{\mu}_{1},\widehat{\mu}_{0})$ converging to $(\mu_{1},\mu_{0})$ and $(\widehat{m}_{1},\widehat{m}_{0})$ to $(m_{1}^{*},m_{0}^{*})$, $\widehat{\boldsymbol{c}}$ converges to $\boldsymbol{c}^{*}=(c_{1}^{*},\dots,c_{k}^{*})$, where

[TABLE]

It can be shown that \sigma_{\textsc{cdr}}^{2}(\boldsymbol{c}^{*}(\cdot;\mbox{\boldmath{\eta}}),\mbox{\boldmath{\eta}}) is minimized by setting ${\eta}$ equal to \mbox{\boldmath{\eta}}_{\text{opt}}^{*}=(\eta_{1,\text{opt}}^{*},\dots,\eta_{k,\text{opt}}^{*}), where, for each $s\in\{1,\dots,k\}$,

[TABLE]

and $\boldsymbol{O}_{s}^{*}=(\boldsymbol{W},A_{s}^{*},Y_{s}^{*})$ is similar to $\boldsymbol{O}_{s}$ with $p_{s}^{*}$ replacing $p_{s}$. For any $(m_{1}^{*},m_{0}^{*})$, \mbox{\boldmath{\eta}}_{\text{opt}}^{*} is consistently estimated by \widehat{}\mbox{\boldmath{\eta}}=(\widehat{\eta}_{1},\dots,\widehat{\eta}_{k}), where, for each $s\in\{1,\dots,k\}$,

[TABLE]

Based on the above optimality results, we propose to estimate $\delta$ using \widehat{\delta}^{\text{aipw}}(\widehat{\boldsymbol{c}}(\cdot;\widehat{}\mbox{\boldmath{\eta}}),\widehat{}\mbox{\boldmath{\eta}}), which is asymptotically equivalent to \widehat{\delta}^{\text{aipw}}(\boldsymbol{c}^{*}(\cdot;\mbox{\boldmath{\eta}}_{\text{opt}}^{*}),\mbox{\boldmath{\eta}}_{\text{opt}}^{*}). This estimator attains the smallest asymptotic variance among all estimators of the form \widehat{\delta}^{\text{aipw}}(\widehat{\boldsymbol{c}}(\cdot;\mbox{\boldmath{\eta}}),\mbox{\boldmath{\eta}}) and, if $(m_{1}^{*},m_{0}^{*})=(m_{1},m_{0})$, among all AIPW estimators considered here. Its asymptotic variance is consistently estimated by

[TABLE]

Appendix C: Proofs

Proof of Theorem 1

Using the Taylor expansions of the function $g(\cdot)$ at $\mu_{1}$ and $\mu_{0}$, we have

[TABLE]

where

[TABLE]

The stage-specific treatment group averages for $\mu_{1}$ and $\mu_{0}$ are given by

[TABLE]

The law of large numbers ensures that, as $n_{1}\rightarrow\infty$,

[TABLE]

Then we have

[TABLE]

[TABLE]

where

[TABLE]

The central limit theorem ensures that, as $n_{1}\rightarrow\infty$,

[TABLE]

Conditioning on the stage 1 data $\boldsymbol{D}_{1}$, $n_{2}$ and $\pi_{2}$ are known. Then we have

[TABLE]

Substituting these into the expression of $Z_{2}$ yields

[TABLE]

where

[TABLE]

Applying the central limit theorem yields

[TABLE]

Moreover, we observe that

[TABLE]

By (A.2), we have $\operatorname{P}(\sqrt{n_{2}}Z_{2}\leq z_{2}|\boldsymbol{D}_{1})\rightarrow\Phi(z_{2}/\sqrt{\operatorname{var}\{\psi_{2}^{\text{aug}}(\boldsymbol{O}_{2})\}})$ as $n_{2}\rightarrow\infty$, where $\Phi(\cdot)$ is the cumulative distribution function of the standard normal distribution. Further letting $n_{1}\rightarrow\infty$, we can obtain that $\operatorname{P}(\sqrt{n_{1}}Z_{1}\leq z_{1})\rightarrow\Phi(z_{1}/\sqrt{\operatorname{var}\{\psi_{1}^{\text{aug}}(\boldsymbol{O}_{1})\}})$ and $\Phi(z_{2}/\sqrt{\operatorname{var}\{\psi_{2}^{\text{aug}}(\boldsymbol{O}_{2})\}})\rightarrow\Phi(z_{2}/\sqrt{\operatorname{var}\{\psi_{2}^{\text{aug}}(\boldsymbol{O}_{2}^{*})\}})$, where

[TABLE]

and $\boldsymbol{O}_{2}^{*}=(\boldsymbol{W},A_{2}^{*},Y_{2}^{*})$ is similar to $\boldsymbol{O}_{2}$ with $\pi_{2}^{*}$ replacing $\pi_{2}$, $\operatorname{P}\{A_{2}^{*}=1|\boldsymbol{W},Y(1),Y(0)\}=\operatorname{P}(A_{2}^{*}=1)=\pi_{2}^{*}$, and $Y_{2}^{*}=Y(A_{2}^{*})=A_{2}^{*}Y(1)+(1-A_{2}^{*})Y(0)$. Since $n_{1}/n_{2}\rightarrow\lambda$ as $n_{1}\rightarrow\infty$, then we have

[TABLE]

as $n_{1}\rightarrow\infty$. This implies that, as $n_{1}\rightarrow\infty$,

[TABLE]

Applying the delta method yields

[TABLE]

The Slutsky’s theorem ensures that

[TABLE]

Derivation of $(b_{1,\text{opt}},b_{2,\text{opt}})$

Note that $\operatorname{var}\{\psi_{1}^{\text{aug}}(\boldsymbol{O}_{1})\}$ (rsp. $\operatorname{var}\{\psi_{2}^{\text{aug}}(\boldsymbol{O}_{2}^{*})\}$) is a function of $b_{1}$ (rsp. $b_{2}$) only. Minimizing the asymptotic variance $\sigma_{\textsc{cir}}^{2}(b_{1},b_{2},\theta)=\operatorname{var}\{\psi_{1}^{\text{aug}}(\boldsymbol{O}_{1})\}+\lambda\operatorname{var}\{\psi_{2}^{\text{aug}}(\boldsymbol{O}_{2}^{*})\}$ with respect to $b_{1}$ (rsp. $b_{2}$) is equivalent to minimizing $\operatorname{var}\{\psi_{1}^{\text{aug}}(\boldsymbol{O}_{1})\}$ (rsp. $\operatorname{var}\{\psi_{2}^{\text{aug}}(\boldsymbol{O}_{2}^{*})\}$) with respect to $b_{1}$ (rsp. $b_{2}$). Simple algebra leads to

[TABLE]

The minimizer of $\operatorname{var}\{\psi_{1}^{\text{aug}}(\boldsymbol{O}_{1})\}$ with respect to $b_{1}$ is easily found to be

[TABLE]

where $m_{a}(\boldsymbol{W})=\operatorname{E}\{Y(a)|\boldsymbol{W}\}=\operatorname{E}(Y|\boldsymbol{W},A=a)$, $a=0,1$. Similarly, the minimizer of $\operatorname{var}\{\psi_{2}^{\text{aug}}(\boldsymbol{O}_{2}^{*})\}$ with respect to $b_{2}$ can be obtained as

[TABLE]

Derivation of $\theta_{\text{opt}}^{*}$

Substituting $(b_{1}^{*},b_{2}^{*})$ into $\sigma_{\textsc{cir}}^{2}(b_{1},b_{2},\theta)$ yields

[TABLE]

where

[TABLE]

The minimizer of $\sigma_{\textsc{cir}}^{2}(b_{1}^{*},b_{2}^{*},\theta)$ with respect to $\theta$ is then equal to

[TABLE]

Convergence of $(\widehat{m}_{1},\widehat{m}_{0})$

Suppose we specify a regression model for the conditional mean, i.e., $\operatorname{E}(Y_{1}|A_{1}=a,\boldsymbol{W})=m_{a}(\boldsymbol{W};\xi)$. The parameter $\xi$ is usually estimated by solving an estimating equation:

[TABLE]

where $h$ is an estimating function of the same dimension as $\xi$. Let $\widehat{\xi}$ denote the resulting estimator, which may be the maximum likelihood estimator for logistic regression (used in our simulation study). It can be argued as in the proof of Theorem 1 that $n^{-1}\sum_{i=1}^{n_{1}}h(\boldsymbol{O}_{1i};\xi)+n^{-1}\sum_{i=1}^{n_{2}}h(\boldsymbol{O}_{2i};\xi)$ converges in probability to

[TABLE]

Under certain regularity conditions (e.g., if $h$ belongs to a Glivenko-Cantelli class; see van der Vaart (1998, Chapter 19)), the convergence in probability is uniform. Let $\xi^{*}$ be the solution to the equation $\lambda/(1+\lambda)\operatorname{E}\{h(\boldsymbol{O}_{1};\xi)\}+1/(1+\lambda)\operatorname{E}\{h(\boldsymbol{O}_{2}^{*};\xi)\}=0$. By Theorem 5.9 of van der Vaart (1998), $\widehat{\xi}$ converges in probability to $\xi^{*}$, and therefore $\widehat{m}_{a}=m_{a}(\cdot;\widehat{\xi})$ converges to $m_{a}^{*}=m_{a}(\cdot;\xi^{*})$.

Consistency of $\widehat{\theta}$

The consistency of $\widehat{\theta}$ follows from the consistency of the variance estimators $\widehat{\sigma}_{1,\textsc{cir}}^{2}$ and $\widehat{\sigma}_{2,\textsc{cir}}^{2}$. First, since the stage 1 data $\boldsymbol{D}_{1}$ are independent and identically distributed, the sample variance estimator $\widehat{\sigma}_{1,\textsc{cir}}^{2}$ converges in probability to $\sigma_{1,\textsc{cir}}^{*2}$ by the law of large numbers. For Stage 2, note that for any $\varepsilon>0$,

[TABLE]

where $\sigma_{2,\textsc{cir}}^{2}$ is defined analogously to $\sigma_{2,\textsc{cir}}^{*2}$ as

[TABLE]

but with $\operatorname{P}\{A_{2}=1|\boldsymbol{W},Y(1),Y(0)\}=\operatorname{P}(A_{2}=1)=\pi_{2}$.

For the first term, by the law of total probability, $\operatorname{P}(|\widehat{\sigma}_{2,\textsc{cir}}^{2}-\sigma_{2,\textsc{cir}}^{2}|>\varepsilon/2)=\operatorname{E}\{\operatorname{P}(|\widehat{\sigma}_{2,\textsc{cir}}^{2}-\sigma_{2,\textsc{cir}}^{2}|>\varepsilon/2\mid\pi_{2})\}$. The inner probability $\operatorname{P}(|\widehat{\sigma}_{2,\textsc{cir}}^{2}-\sigma_{2,\textsc{cir}}^{2}|>\varepsilon/2\mid\pi_{2})$ converges to zero as $n_{2}\rightarrow\infty$, since conditional on $\pi_{2}$, the stage 2 data $\boldsymbol{D}_{2}$ are independent and identically distributed. Therefore, by the bounded convergence theorem, the first term converges to zero. For the second term, note that $\sigma_{2,\textsc{cir}}^{2}$ and $\sigma_{2,\textsc{cir}}^{*2}$ are continuous functionals of $\pi_{2}$ and $\pi_{2}^{*}$, respectively. Under the assumption that $\pi_{2}$ converges in probability to $\pi_{2}^{*}$, the second term converges to zero as $n_{1}\to\infty$ by the continuous mapping theorem. Combining these results implies that $\widehat{\sigma}_{2,\textsc{cir}}^{2}$ converges in probability to $\sigma_{2,\textsc{cir}}^{*2}$. Consequently, by Slutsky’s theorem, the estimator $\hat{\theta}$ is consistent for $\theta_{\text{opt}}^{*}$.

Proof of Theorem 2

According to the proof of Theorem 1, we can write

[TABLE]

[TABLE]

The condition $\widetilde{\theta}\rightarrow\theta$ as $n_{1}\rightarrow\infty$ ensures that

[TABLE]

Moreover, by the Chebyshev inequality (Shao, 2003), for any real number $\varepsilon>0$, we have

[TABLE]

where the second inequality is due to $|A_{1i}-\pi_{1}|\leq 1$ and the last equality is due to the condition $\|\widetilde{b}_{1}-b_{1}^{*}\|_{2}=[\operatorname{E}\{\widetilde{b}_{1}-b_{1}^{*}\}^{2}]^{1/2}=o_{p}(1)$, so that

[TABLE]

Conditioning on $\boldsymbol{D}_{1}$, $n_{2}$ and $\pi_{2}$ are known. Then we can similarly obtain that

[TABLE]

and

[TABLE]

Therefore, we have

[TABLE]

where

[TABLE]

and the second equality is due to $\sqrt{n_{1}}(\widehat{\mu}_{1}-\mu_{1})=O_{p}(1)$ and $\sqrt{n_{1}}(\widehat{\mu}_{0}-\mu_{0})=O_{P}(1)$ by the central limit theorem. Following the proofs of Theorem 1, we can show that

[TABLE]

where

[TABLE]

and $\boldsymbol{O}_{2}^{*}=(\boldsymbol{W},A_{2}^{*},Y_{2}^{*})$ is similar to $\boldsymbol{O}_{2}$ with $\pi_{2}^{*}$ replacing $\pi_{2}$ and $\operatorname{P}\{A_{2}^{*}=1|\boldsymbol{W},Y(1),Y(0)\}=\operatorname{P}(A_{2}^{*}=1)=\pi_{2}^{*}$. Applying the Slutsky’s theorem yields

[TABLE]

Proof of Theorem S1

Expanding the function $g(\cdot)$ at $\mu_{1}$ and $\mu_{0}$ yields

[TABLE]

where

[TABLE]

Recall that

[TABLE]

By the law of large numbers, as $n_{1}\rightarrow\infty$, we have

[TABLE]

so that

[TABLE]

Then we can write

[TABLE]

where

[TABLE]

The central limit theorem ensures that, as $n_{1}\rightarrow\infty$,

[TABLE]

By conditioning on the stage 1 data $\boldsymbol{D}_{1}$, we have

[TABLE]

as $n_{2}\rightarrow\infty$. Then $H_{2}$ can be written as

[TABLE]

where

[TABLE]

By the central limit theorem, we have

[TABLE]

Moreover, we observe that

[TABLE]

By (A.4), as $n_{2}\rightarrow\infty$, $\operatorname{P}(\sqrt{n_{2}}H_{2}\leq h_{2}|\boldsymbol{D}_{1})\rightarrow\Phi(h_{2}/\sqrt{\operatorname{var}\{\psi_{2}^{\text{aipw}}(\boldsymbol{O}_{2})\}})$. Further letting $n_{1}\rightarrow\infty$, we have $\operatorname{P}(\sqrt{n_{1}}H_{1}\leq h_{1})\rightarrow\Phi(h_{1}/\sqrt{\operatorname{var}\{\psi_{1}^{\text{aipw}}(\boldsymbol{O}_{1})\}})$ and $\Phi(h_{2}/\sqrt{\operatorname{var}\{\psi_{2}^{\text{aipw}}(\boldsymbol{O}_{2})\}})\rightarrow\Phi(h_{2}/\sqrt{\operatorname{var}\{\psi_{2}^{\text{aipw}}(\boldsymbol{O}_{2}^{*})\}})$, where

[TABLE]

and $\boldsymbol{O}_{2}^{*}=(\boldsymbol{W},A_{2}^{*},Y_{2}^{*})$ is similar to $\boldsymbol{O}_{2}$ with $p_{2}^{*}$ replacing $p_{2}$, $\operatorname{P}\{A_{2}^{*}=1|\boldsymbol{W},Y(1),Y(0)\}=\operatorname{P}(A_{2}^{*}=1|\boldsymbol{W})=p_{2}^{*}(\boldsymbol{W})$, and $Y_{2}^{*}=Y(A_{2}^{*})=A_{2}^{*}Y(1)+(1-A_{2}^{*})Y(0)$. This, together with the condition that $n_{1}/n_{2}\rightarrow\lambda$ as $n_{1}\rightarrow\infty$, yields

[TABLE]

It thus follows that

[TABLE]

The delta method ensures that

[TABLE]

and hence

[TABLE]

Derivation of $(c_{1,\text{opt}},c_{2,\text{opt}})$

The asymptotic variance of the AIPW estimator $\widehat{\delta}^{\text{aipw}}(c_{1},c_{2},\eta)$ is

[TABLE]

We observe that $\operatorname{var}\{\psi_{1}^{\text{aipw}}(\boldsymbol{O}_{1})\}$ (rsp. $\operatorname{var}\{\psi_{2}^{\text{aipw}}(\boldsymbol{O}_{2}^{*})\}$) only depends on $c_{1}$ (rsp. $c_{2}$). To minimize $\operatorname{var}\{\psi_{1}^{\text{aipw}}(\boldsymbol{O}_{1})\}+\lambda\operatorname{var}\{\psi_{2}^{\text{aipw}}(\boldsymbol{O}_{2}^{*})\}$ with respect to $c_{1}$ (rsp. $c_{2}$), it suffices to consider $\operatorname{var}\{\psi_{1}^{\text{aipw}}(\boldsymbol{O}_{1})\}$ (rsp. $\operatorname{var}\{\psi_{2}^{\text{aipw}}(\boldsymbol{O}_{2};p_{2}^{*})\}$). By conditioning on $\boldsymbol{W}$, we can write

[TABLE]

The minimizer of $\operatorname{var}\{\psi_{1}^{\text{aipw}}(\boldsymbol{O}_{1})\}$ with respect to $c_{1}$ is found to be

[TABLE]

Moreover, by conditioning on $\boldsymbol{W}$, the variance of $\psi_{2}^{\text{aipw}}(\boldsymbol{O}_{2}^{*})$ is equal to

[TABLE]

The minimizer of the preceding display with respect to $c_{2}$ is given by

[TABLE]

Derivation of $\eta_{\text{opt}}^{*}$

Substituting $(c_{1}^{*},c_{2}^{*})$ into $\psi_{1}^{\text{aipw}}(\boldsymbol{O}_{1})$ and $\psi_{2}^{\text{aipw}}(\boldsymbol{O}_{2}^{*})$ yields

[TABLE]

The asymptotic variance $\sigma_{\textsc{cdr}}^{2}(c_{1}^{*},c_{2}^{*},\eta)$ can then be written as

[TABLE]

where

[TABLE]

The minimizer of $\operatorname{var}\{\psi_{1}^{\text{aipw}}(\boldsymbol{O}_{1};c_{1}^{*})\}+\lambda\operatorname{var}\{\psi_{2}^{\text{aipw}}(\boldsymbol{O}_{2}^{*};c_{2}^{*})\}$ with respect to $\eta$ is given by

[TABLE]

Proof of Theorem S2

According to the proofs of Theorem S1, we can write

[TABLE]

The condition $\widetilde{\eta}\rightarrow\eta$ as $n_{1}\rightarrow\infty$ ensures that

[TABLE]

By the chebyshev’s inequality, for any real number $\varepsilon$, we have

[TABLE]

where the second inequality is because $|A_{1i}-p_{1}(\boldsymbol{W}_{1i})|\leq 1$ and the last equality is because the condition $\|\widetilde{c}_{1}-c_{1}\|_{2}=[\operatorname{E}\{\widetilde{c}_{1}-c_{1}\}^{2}]^{1/2}=o_{p}(1)$, Hence

[TABLE]

By conditioning on the stage 1 data $\boldsymbol{D}_{1}$, we can similarly obtain that

[TABLE]

Combining these results leads to

[TABLE]

where

[TABLE]

Following the proofs of Theorem S1, we can show that

[TABLE]

where

[TABLE]

and $\boldsymbol{O}_{2}^{*}=(\boldsymbol{W},A_{2}^{*},Y_{2}^{*})$ is similar to $\boldsymbol{O}_{2}$ with $p_{2}^{*}$ replacing $p_{2}$, $\operatorname{P}\{A_{2}^{*}=1|\boldsymbol{W},Y(1),Y(0)\}=\operatorname{P}(A_{2}^{*}=1|\boldsymbol{W})=p_{2}^{*}(\boldsymbol{W})$, and $Y_{2}^{*}=Y(A_{2}^{*})=A_{2}^{*}Y(1)+(1-A_{2}^{*})Y(0)$. The conclusion of the theorem thus follows.

Appendix D: Additional Simulations

Smaller sample size

Table S2 reports additional simulation results for a smaller sample size ($n_{1}=n_{0}=150$). Except for the reduced sample size, this simulation study is identical to the one described in Section 5. We will focus on optimized estimators in design comparisons. In Setting 1, the two-stage designs perform similarly to each other and generally better than the one-stage design. This suggests that stage 1 data with $n_{1}=150$ may be insufficient for estimating the optimal CDR design. In Setting 2, the results for $n_{1}=n_{0}=150$ are similar to those for $n_{1}=n_{0}=250$. Specifically, the one-stage CDR design continues to underperform the one-stage CIR design, and the two-stage designs consistently outperform the one-stage designs. Among the three two-stage designs, the two-stage hybrid design generally achieves the highest efficiency, although the incremental improvement tends to be small.

Allocation of stage-specific sample sizes

Here we report a simulation study to evaluate how the sample size allocation between stages affects the performance of adaptive designs. Fixing the total sample size at $n=500$, we consider two alternative proportions for stage 1 sample size: $n_{1}/n=30\%$ and $n_{1}/n=70\%$, in addition to the original scenario with $n_{1}/n=50\%$. In all other aspects, this simulation study is identical to Setting 1 of the main simulation study (described in Section 5.1).

Table S3 reports relative efficiency results with the optimized estimator under the one-stage design as the reference. When $n_{1}/n=30\%$, the two-stage CIR and CDR designs outperform the one-stage design, with the two-stage CIR design achieving the highest efficiency. This suggests that a stage 1 sample size of $n_{1}=150$ may be insufficient to fully optimize the CDR design. When $n_{1}/n=70\%$, the two-stage CIR and CDR designs perform similarly and both remain more efficient than the one-stage design. Although a larger stage 1 sample size ($n_{1}=350$) improves estimation of the optimal CDR design, the reduced Stage 2 sample limits the impact of the estimated optimal design. These results highlight the trade-off between allocating sufficient stage 1 data for optimizing treatment allocation and retaining adequate Stage 2 data for precise treatment effect estimation.

Non-prognostic covariates

Here, we present a simulation study in situations where some baseline covariates are not prognostic. Following Setting 1 in Section 5.1 of the main text, we generate the potential outcomes using the logistic regression model \operatorname{logit}[\operatorname{P}\{Y(a)=1|\boldsymbol{W}\}]=\gamma_{0}+\gamma_{1}a+\mbox{\boldmath{\gamma}}_{2}^{\prime}\boldsymbol{W}+\mbox{\boldmath{\gamma}}_{3}^{\prime}(a\boldsymbol{W}), where $\gamma_{0}=-2.5$ and $\gamma_{1}=1$ or $2$, for $a=0,1$. We consider two scenarios that differ in the number of prognostic covariates: (1) one prognostic covariate, with \mbox{\boldmath{\gamma}}_{2}=(0,0,0.2)^{\prime} and \mbox{\boldmath{\gamma}}_{3}=(0,0,-1.5)^{\prime}; and (2) two prognostic covariates, with \mbox{\boldmath{\gamma}}_{2}=(0,-0.2,0.2)^{\prime} and \mbox{\boldmath{\gamma}}_{3}=(0,-1,-1.5)^{\prime}. In all other aspects, this simulation study is identical to Setting 1 of the main study.

Table S4 reports the relative efficiency results using the optimized estimator under the one-stage CIR design as the reference. The results resemble those shown in the upper section of Table 1 in the main text. The two-stage CIR and CDR designs are generally more efficient than the one-stage design. The efficiency gains from the two-stage designs tend to increase with the number of prognostic covariates, though the improvements are small. When $\boldsymbol{X}$ includes at least one prognostic covariate, the two-stage CDR design typically achieves the highest efficiency among the three designs. In cases where $\boldsymbol{X}$ is not prognostic for either treatment, for example, in Scenario (1) with $\boldsymbol{X}=W_{1},W_{2}$ or $(W_{1},W_{2})^{\prime}$, the two-stage CIR and CDR designs perform similarly. In these cases, the optimal treatment allocation reduces to the Neyman allocation for both CIR and CDR, which explains why the two-stage designs perform similarly to each other and better than the one-stage design.

More severe model misspecification

The model misspecification in the main simulation study is somewhat moderate. To investigate the impact of more severe misspecification, we have conducted an additional simulation study, which is identical to Setting 1 of the main study except for a different working model for design optimization: \operatorname{logit}\{\operatorname{P}(Y_{1}=1|A_{1},\boldsymbol{X})\}=\alpha_{0}+\alpha_{1}A_{1}+\mbox{\boldmath{\alpha}}_{2}^{\prime}\boldsymbol{X}. This working model is obviously more severely misspecified than the original model due to the omission of the treatment-by-covariate interaction terms. The results of this additional study, shown in Table S5, are generally consistent with the original results in that the two-stage CIR and CDR designs continue to outperform the one-stage design. A notable difference is that the efficiency advantage of the two-stage CDR design over the two-stage CIR design appears to have diminished, particularly when the dimension of $\boldsymbol{X}$ is larger (2 or 3). The two-stage CIR design appears more robust against this more severe form of misspecification.