Doubly Robust Difference-in-Differences Estimators

Pedro H. C. Sant'Anna; Jun B. Zhao

arXiv:1812.01723·econ.EM·May 7, 2020

Doubly Robust Difference-in-Differences Estimators

Pedro H. C. Sant'Anna, Jun B. Zhao

PDF

TL;DR

This paper introduces doubly robust estimators for the ATT in DID designs, which are consistent if either the propensity score or outcome model is correct, and attain efficiency bounds under correct specification.

Contribution

It develops novel doubly robust estimators for DID that improve robustness and efficiency, with theoretical derivations and practical implementation tools.

Findings

01

Estimators are consistent if either model is correct.

02

They attain the semiparametric efficiency bound when models are correct.

03

Simulation and empirical results show good finite-sample performance.

Abstract

This article proposes doubly robust estimators for the average treatment effect on the treated (ATT) in difference-in-differences (DID) research designs. In contrast to alternative DID estimators, the proposed estimators are consistent if either (but not necessarily both) a propensity score or outcome regression working models are correctly specified. We also derive the semiparametric efficiency bound for the ATT in DID designs when either panel or repeated cross-section data are available, and show that our proposed estimators attain the semiparametric efficiency bound when the working models are correctly specified. Furthermore, we quantify the potential efficiency gains of having access to panel data instead of repeated cross-section data. Finally, by paying articular attention to the estimation method used to estimate the nuisance parameters, we show that one can sometimes construct…

Equations174

P (Y \leq y, D = d, X \leq x, T = t)

P (Y \leq y, D = d, X \leq x, T = t)

τ = E [Y_{i 1} (1) - Y_{i 1} (0) ∣ D_{i} = 1] .

τ = E [Y_{i 1} (1) - Y_{i 1} (0) ∣ D_{i} = 1] .

τ = E [Y_{1} (1) ∣ D = 1] - E [Y_{1} (0) ∣ D = 1] = E [Y_{1} ∣ D = 1] - E [Y_{1} (0) ∣ D = 1],

τ = E [Y_{1} (1) ∣ D = 1] - E [Y_{1} (0) ∣ D = 1] = E [Y_{1} ∣ D = 1] - E [Y_{1} (0) ∣ D = 1],

τ^{r e g} = \overset{ˉ}{Y}_{1, 1} - \overset{ˉ}{Y}_{1, 0} + n_{t r e a t}^{- 1} i ∣ D_{i} = 1 \sum (μ_{0, 1} (X_{i}) - μ_{0, 0} (X_{i})),

τ^{r e g} = \overset{ˉ}{Y}_{1, 1} - \overset{ˉ}{Y}_{1, 0} + n_{t r e a t}^{- 1} i ∣ D_{i} = 1 \sum (μ_{0, 1} (X_{i}) - μ_{0, 0} (X_{i})),

τ = \frac{1}{E [ D ]} \leavevmode E [\frac{D - p ( X )}{1 - p ( X )} (Y_{1} - Y_{0})]

τ = \frac{1}{E [ D ]} \leavevmode E [\frac{D - p ( X )}{1 - p ( X )} (Y_{1} - Y_{0})]

τ = \frac{1}{E [ D ]} \leavevmode E [\frac{D - p ( X )}{1 - p ( X )} \frac{T - λ}{λ ( 1 - λ )} Y]

τ = \frac{1}{E [ D ]} \leavevmode E [\frac{D - p ( X )}{1 - p ( X )} \frac{T - λ}{λ ( 1 - λ )} Y]

τ^{i pw, p} = \frac{1}{E _{n} [ D ]} \leavevmode E_{n} [\frac{D - π ( X )}{1 - π ( X )} (Y_{1} - Y_{0})],

τ^{i pw, p} = \frac{1}{E _{n} [ D ]} \leavevmode E_{n} [\frac{D - π ( X )}{1 - π ( X )} (Y_{1} - Y_{0})],

Y_{i t} = α_{1} + α_{2} \leavevmode T_{i} + α_{3} \leavevmode D_{i} + τ^{f e} \leavevmode (T_{i} \cdot D_{i}) + θ^{'} X_{i} + ϵ_{i t},

Y_{i t} = α_{1} + α_{2} \leavevmode T_{i} + α_{3} \leavevmode D_{i} + τ^{f e} \leavevmode (T_{i} \cdot D_{i}) + θ^{'} X_{i} + ϵ_{i t},

τ^{d r, p} = E [(w_{1}^{p} (D) - w_{0}^{p} (D, X; π)) (Δ Y - μ_{0, Δ}^{p} (X))],

τ^{d r, p} = E [(w_{1}^{p} (D) - w_{0}^{p} (D, X; π)) (Δ Y - μ_{0, Δ}^{p} (X))],

w_{1}^{p} (D) = \frac{D}{E [ D ]}, \leavevmode \leavevmode \leavevmode \leavevmode and \leavevmode \leavevmode \leavevmode w_{0}^{p} (D, X; g) = \frac{g ( X ) ( 1 - D )}{1 - g ( X )} / E [\frac{g ( X ) ( 1 - D )}{1 - g ( X )}] .

w_{1}^{p} (D) = \frac{D}{E [ D ]}, \leavevmode \leavevmode \leavevmode \leavevmode and \leavevmode \leavevmode \leavevmode w_{0}^{p} (D, X; g) = \frac{g ( X ) ( 1 - D )}{1 - g ( X )} / E [\frac{g ( X ) ( 1 - D )}{1 - g ( X )}] .

τ_{1}^{d r, r c} = E [(w_{1}^{r c} (D, T) - w_{0}^{r c} (D, T, X; π)) (Y - μ_{0, Y}^{r c} (T, X))],

τ_{1}^{d r, r c} = E [(w_{1}^{r c} (D, T) - w_{0}^{r c} (D, T, X; π)) (Y - μ_{0, Y}^{r c} (T, X))],

τ_{2}^{d r, r c} = τ_{1}^{d r, r c} + (E [μ_{1, 1}^{r c} (X) - μ_{0, 1}^{r c} (X) D = 1] - E [μ_{1, 1}^{r c} (X) - μ_{0, 1}^{r c} (X) D = 1, T = 1]) - (E [μ_{1, 0}^{r c} (X) - μ_{0, 0}^{r c} (X) D = 1] - E [μ_{1, 0}^{r c} (X) - μ_{0, 0}^{r c} (X) D = 1, T = 0]),

τ_{2}^{d r, r c} = τ_{1}^{d r, r c} + (E [μ_{1, 1}^{r c} (X) - μ_{0, 1}^{r c} (X) D = 1] - E [μ_{1, 1}^{r c} (X) - μ_{0, 1}^{r c} (X) D = 1, T = 1]) - (E [μ_{1, 0}^{r c} (X) - μ_{0, 0}^{r c} (X) D = 1] - E [μ_{1, 0}^{r c} (X) - μ_{0, 0}^{r c} (X) D = 1, T = 0]),

w_{1}^{r c} (D, T) = w_{1, 1}^{r c} (D, T) - w_{1, 0}^{r c} (D, T), \leavevmode \leavevmode \leavevmode \leavevmode and \leavevmode \leavevmode \leavevmode w_{0}^{r c} (D, T, X; g) = w_{0, 1}^{r c} (D, T, X; g) - w_{0, 0}^{r c} (D, T, X; g),

w_{1}^{r c} (D, T) = w_{1, 1}^{r c} (D, T) - w_{1, 0}^{r c} (D, T), \leavevmode \leavevmode \leavevmode \leavevmode and \leavevmode \leavevmode \leavevmode w_{0}^{r c} (D, T, X; g) = w_{0, 1}^{r c} (D, T, X; g) - w_{0, 0}^{r c} (D, T, X; g),

w_{1, t}^{r c} (D, T)

w_{1, t}^{r c} (D, T)

w_{0, t}^{r c} (D, T, X; g)

η^{e, p} (Y_{1}, Y_{0}, D, X) = \leavevmode w_{1}^{p} (D) (m_{1, Δ}^{p} (X) - m_{0, Δ}^{p} (X) - τ) + w_{1}^{p} (D) (Δ Y - m_{1, Δ}^{p} (X)) - w_{0}^{p} (D, X; p) (Δ Y - m_{0, Δ}^{p} (X)),

η^{e, p} (Y_{1}, Y_{0}, D, X) = \leavevmode w_{1}^{p} (D) (m_{1, Δ}^{p} (X) - m_{0, Δ}^{p} (X) - τ) + w_{1}^{p} (D) (Δ Y - m_{1, Δ}^{p} (X)) - w_{0}^{p} (D, X; p) (Δ Y - m_{0, Δ}^{p} (X)),

E [η^{e, p} (Y_{1}, Y_{0}, D, X)^{2}] = \frac{1}{E [ D ] ^{2}} E [D (m_{1, Δ}^{p} (X) - m_{0, Δ}^{p} (X) - τ)^{2} + D (Δ Y - m_{1, Δ}^{p} (X))^{2} + \frac{( 1 - D ) p ( X ) ^{2}}{( 1 - p ( X ) ) ^{2}} (Δ Y - m_{0, Δ}^{p} (X))^{2}] .

E [η^{e, p} (Y_{1}, Y_{0}, D, X)^{2}] = \frac{1}{E [ D ] ^{2}} E [D (m_{1, Δ}^{p} (X) - m_{0, Δ}^{p} (X) - τ)^{2} + D (Δ Y - m_{1, Δ}^{p} (X))^{2} + \frac{( 1 - D ) p ( X ) ^{2}}{( 1 - p ( X ) ) ^{2}} (Δ Y - m_{0, Δ}^{p} (X))^{2}] .

η^{e, r c} (Y, D, T, X) = \leavevmode \frac{D}{E [ D ]} (m_{1, Δ}^{r c} (X) - m_{0, Δ}^{r c} (X) - τ) + (w_{1, 1}^{r c} (D, T) (Y - m_{1, 1}^{r c} (X)) - w_{1, 0}^{r c} (D, T) (Y - m_{1, 0}^{r c} (X))) - (w_{0, 1}^{r c} (D, T, X; p) (Y - m_{0, 1}^{r c} (X)) - w_{0, 0}^{r c} (D, T, X; p) (Y - m_{0, 0}^{r c} (X))),

η^{e, r c} (Y, D, T, X) = \leavevmode \frac{D}{E [ D ]} (m_{1, Δ}^{r c} (X) - m_{0, Δ}^{r c} (X) - τ) + (w_{1, 1}^{r c} (D, T) (Y - m_{1, 1}^{r c} (X)) - w_{1, 0}^{r c} (D, T) (Y - m_{1, 0}^{r c} (X))) - (w_{0, 1}^{r c} (D, T, X; p) (Y - m_{0, 1}^{r c} (X)) - w_{0, 0}^{r c} (D, T, X; p) (Y - m_{0, 0}^{r c} (X))),

E [η^{e, r c} (Y, D, T, X)^{2}] = \frac{1}{E [ D ] ^{2}} E [D (m_{1, Δ}^{r c} (X) - m_{0, Δ}^{r c} (X) - τ)^{2} + \frac{D T}{λ ^{2}} (Y - m_{1, 1}^{r c} (X))^{2} + \frac{D ( 1 - T )}{( 1 - λ ) ^{2}} (Y - m_{1, 0}^{r c} (X))^{2} + \frac{( 1 - D ) p ( X ) ^{2} T}{( 1 - p ( X ) ) ^{2} λ ^{2}} (Y - m_{0, 1}^{r c} (X))^{2} + \frac{( 1 - D ) p ( X ) ^{2} ( 1 - T )}{( 1 - p ( X ) ) ^{2} ( 1 - λ ) ^{2}} (Y - m_{0, 0}^{r c} (X))^{2}] .

E [η^{e, r c} (Y, D, T, X)^{2}] = \frac{1}{E [ D ] ^{2}} E [D (m_{1, Δ}^{r c} (X) - m_{0, Δ}^{r c} (X) - τ)^{2} + \frac{D T}{λ ^{2}} (Y - m_{1, 1}^{r c} (X))^{2} + \frac{D ( 1 - T )}{( 1 - λ ) ^{2}} (Y - m_{1, 0}^{r c} (X))^{2} + \frac{( 1 - D ) p ( X ) ^{2} T}{( 1 - p ( X ) ) ^{2} λ ^{2}} (Y - m_{0, 1}^{r c} (X))^{2} + \frac{( 1 - D ) p ( X ) ^{2} ( 1 - T )}{( 1 - p ( X ) ) ^{2} ( 1 - λ ) ^{2}} (Y - m_{0, 0}^{r c} (X))^{2}] .

η^{e, p} (Y_{1}, Y_{0}, D, X) = (w_{1}^{p} (D) - w_{0}^{p} (D, X; p)) (Δ Y - m_{0, Δ} (X)) - w_{1}^{p} (D) \cdot τ,

η^{e, p} (Y_{1}, Y_{0}, D, X) = (w_{1}^{p} (D) - w_{0}^{p} (D, X; p)) (Δ Y - m_{0, Δ} (X)) - w_{1}^{p} (D) \cdot τ,

E [η^{e, r c} (Y, D, T, X)^{2}] - E [η^{e, p} (Y_{1}, Y_{0}, D, X)^{2}] = \frac{1}{E [ D ] ^{2}} E D (\frac{1 - λ}{λ} (Y_{1} - m_{1, 1} (X)) + \frac{λ}{1 - λ} (Y_{0} - m_{1, 0} (X)))^{2} + \frac{( 1 - D ) p ( X ) ^{2}}{( 1 - p ( X ) ) ^{2}} (\frac{1 - λ}{λ} (Y_{1} - m_{0, 1} (X)) + \frac{λ}{1 - λ} (Y_{0} - m_{0, 0} (X)))^{2} \geq 0.

E [η^{e, r c} (Y, D, T, X)^{2}] - E [η^{e, p} (Y_{1}, Y_{0}, D, X)^{2}] = \frac{1}{E [ D ] ^{2}} E D (\frac{1 - λ}{λ} (Y_{1} - m_{1, 1} (X)) + \frac{λ}{1 - λ} (Y_{0} - m_{1, 0} (X)))^{2} + \frac{( 1 - D ) p ( X ) ^{2}}{( 1 - p ( X ) ) ^{2}} (\frac{1 - λ}{λ} (Y_{1} - m_{0, 1} (X)) + \frac{λ}{1 - λ} (Y_{0} - m_{0, 0} (X)))^{2} \geq 0.

E [D (Y_{0} - m_{1, 0} (X))^{2} + \frac{( 1 - D ) p ( X ) ^{2}}{( 1 - p ( X ) ) ^{2}} (Y_{0} - m_{0, 0} (X))^{2}] = E [D (Y_{1} - m_{1, 1} (X))^{2} + \frac{( 1 - D ) p ( X ) ^{2}}{( 1 - p ( X ) ) ^{2}} (Y_{1} - m_{0, 1} (X))^{2}],

E [D (Y_{0} - m_{1, 0} (X))^{2} + \frac{( 1 - D ) p ( X ) ^{2}}{( 1 - p ( X ) ) ^{2}} (Y_{0} - m_{0, 0} (X))^{2}] = E [D (Y_{1} - m_{1, 1} (X))^{2} + \frac{( 1 - D ) p ( X ) ^{2}}{( 1 - p ( X ) ) ^{2}} (Y_{1} - m_{0, 1} (X))^{2}],

\tilde{σ}_{t}^{2} = E [D (Y_{t} - m_{1, t} (X))^{2} + \frac{( 1 - D ) p ( X ) ^{2}}{( 1 - p ( X ) ) ^{2}} (Y_{t} - m_{0, t} (X))^{2}] .

\tilde{σ}_{t}^{2} = E [D (Y_{t} - m_{1, t} (X))^{2} + \frac{( 1 - D ) p ( X ) ^{2}}{( 1 - p ( X ) ) ^{2}} (Y_{t} - m_{0, t} (X))^{2}] .

τ^{d r, p} = E_{n} [(w_{1}^{p} (D) - w_{0}^{p} (D, X; γ)) (Δ Y - μ_{0, Δ}^{p} (X; β_{0, 0}^{p}, β_{0, 1}^{p}))],

τ^{d r, p} = E_{n} [(w_{1}^{p} (D) - w_{0}^{p} (D, X; γ)) (Δ Y - μ_{0, Δ}^{p} (X; β_{0, 0}^{p}, β_{0, 1}^{p}))],

w_{1}^{p} (D) = \frac{D}{E _{n} [ D ]}, \leavevmode \nobreak \leavevmode \nobreak \leavevmode \nobreak \leavevmode \nobreak \leavevmode \nobreak and \leavevmode \nobreak \leavevmode \nobreak \leavevmode \nobreak \leavevmode \nobreak \leavevmode \nobreak w_{0}^{p} (D, X; γ) = \frac{π ( X ; γ ) ( 1 - D )}{1 - π ( X ; γ )} / E_{n} [\frac{π ( X ; γ ) ( 1 - D )}{1 - π ( X ; γ ))}],

w_{1}^{p} (D) = \frac{D}{E _{n} [ D ]}, \leavevmode \nobreak \leavevmode \nobreak \leavevmode \nobreak \leavevmode \nobreak \leavevmode \nobreak and \leavevmode \nobreak \leavevmode \nobreak \leavevmode \nobreak \leavevmode \nobreak \leavevmode \nobreak w_{0}^{p} (D, X; γ) = \frac{π ( X ; γ ) ( 1 - D )}{1 - π ( X ; γ )} / E_{n} [\frac{π ( X ; γ ) ( 1 - D )}{1 - π ( X ; γ ))}],

τ_{1}^{d r, r c} = E_{n} [(w_{1}^{r c} (D, T) - w_{0}^{r c} (D, T, X; γ)) (Y - μ_{0, Y}^{r c} (T, X; β_{0, 0}^{r c}, β_{0, 1}^{r c}))],

τ_{1}^{d r, r c} = E_{n} [(w_{1}^{r c} (D, T) - w_{0}^{r c} (D, T, X; γ)) (Y - μ_{0, Y}^{r c} (T, X; β_{0, 0}^{r c}, β_{0, 1}^{r c}))],

τ_{2}^{d r, r c} = τ_{1}^{d r, r c} + (E_{n} [(\frac{D}{E _{n} [ D ]} - w_{1, 1}^{r c} (D, T)) (μ_{1, 1}^{r c} (X; β_{1, 1}^{r c}) - μ_{0, 1}^{r c} (X; β_{0, 1}^{r c}))]) - (E_{n} [(\frac{D}{E _{n} [ D ]} - w_{1, 0}^{r c} (D, T)) (μ_{1, 0}^{r c} (X; β_{1, 0}^{r c}) - μ_{0, 0}^{r c} (X; β_{0, 0}^{r c}))]),

τ_{2}^{d r, r c} = τ_{1}^{d r, r c} + (E_{n} [(\frac{D}{E _{n} [ D ]} - w_{1, 1}^{r c} (D, T)) (μ_{1, 1}^{r c} (X; β_{1, 1}^{r c}) - μ_{0, 1}^{r c} (X; β_{0, 1}^{r c}))]) - (E_{n} [(\frac{D}{E _{n} [ D ]} - w_{1, 0}^{r c} (D, T)) (μ_{1, 0}^{r c} (X; β_{1, 0}^{r c}) - μ_{0, 0}^{r c} (X; β_{0, 0}^{r c}))]),

π (X, γ) = Λ (X^{'} γ) \equiv \frac{exp ( X ^{'} γ )}{1 + exp ( X ^{'} γ )}, and μ_{0, Δ}^{p} (X; β_{0, 1}^{p}, β_{0, 1}^{p}) = μ_{0, Δ}^{l in, p} (X; β_{0, Δ}^{p}) \equiv X^{'} β_{0, Δ}^{p} .

π (X, γ) = Λ (X^{'} γ) \equiv \frac{exp ( X ^{'} γ )}{1 + exp ( X ^{'} γ )}, and μ_{0, Δ}^{p} (X; β_{0, 1}^{p}, β_{0, 1}^{p}) = μ_{0, Δ}^{l in, p} (X; β_{0, Δ}^{p}) \equiv X^{'} β_{0, Δ}^{p} .

τ_{im p}^{d r, p} = E_{n} [(w_{1}^{p} (D) - w_{0}^{p} (D, X; γ^{i pt})) (Δ Y - μ_{0, Δ}^{l in, p} (X; β_{0, Δ}^{w l s, p}))],

τ_{im p}^{d r, p} = E_{n} [(w_{1}^{p} (D) - w_{0}^{p} (D, X; γ^{i pt})) (Δ Y - μ_{0, Δ}^{l in, p} (X; β_{0, Δ}^{w l s, p}))],

γ^{i pt}

γ^{i pt}

β_{0, Δ}^{w l s, p}

Peer Reviews

No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.

Videos

No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.

Full text

\deffootnote

1em1.6em\thefootnotemark

Doubly Robust Difference-in-Differences Estimators††thanks: First complete version: November 29, 2018. We thank the editor, Serena Ng, the

associate editor, two anonymous referees, Brantly Callaway, Alex Poirier, Vitor Possebom, Yuya Sasaki, Tymon Słoczyński, Qi Xu, and the audiences of the 2018 SEA conference, 2019 New York Econometrics Camp, and the 2019 IAAE conference for valuable comments.

Pedro H. C. Sant’Anna

Vanderbilt University Department of Economics, Vanderbilt University. E-mail: [email protected]. Part of this article was written when I was visiting the Cowles Foundation at Yale University, whose hospitality is gratefully acknowledged.

Jun B. Zhao

Vanderbilt University Department of Economics, Vanderbilt University. E-mail: [email protected].

(May 5, 2020)

Abstract

This article proposes doubly robust estimators for the average treatment effect on the treated (ATT) in difference-in-differences (DID) research designs. In contrast to alternative DID estimators, the proposed estimators are consistent if either (but not necessarily both) a propensity score or outcome regression working models are correctly specified. We also derive the semiparametric efficiency bound for the ATT in DID designs when either panel or repeated cross-section data are available, and show that our proposed estimators attain the semiparametric efficiency bound when the working models are correctly specified. Furthermore, we quantify the potential efficiency gains of having access to panel data instead of repeated cross-section data. Finally, by paying particular attention to the estimation method used to estimate the nuisance parameters, we show that one can sometimes construct doubly robust DID estimators for the ATT that are also doubly robust for inference. Simulation studies and an empirical application illustrate the desirable finite-sample performance of the proposed estimators. Open-source software for implementing the proposed policy evaluation tools is available.

1 Introduction

Difference-in-differences (DID) methods are among the most popular procedures practitioners adopted to conduct policy evaluation with observational data. In its canonical form, DID identifies the average treatment effect on the treated (ATT) by comparing the difference in pre and post-treatment outcomes of two groups: one that receives and one that does not receive the treatment (the treated and comparison group, respectively). In order to attach a causal interpretation to DID estimators, researchers routinely invoke the (unconditional) parallel trends assumption (PTA): in the absence of the treatment, the average outcome for the treatment and comparison groups would have followed parallel paths over time. Although the PTA is fundamentally untestable, its plausibility is usually questioned if the observed characteristics that are thought to be associated with the evolution of the outcome are not balanced between the treated and comparison group. In such cases, researchers usually deviate from the canonical DID setup and incorporate pre-treatment covariates into the DID analysis and assume that the PTA is satisfied only after conditioning on these covariates.

In this paper, we study the robustness and efficiency properties of DID estimators for the ATT when the PTA holds after conditioning on covariates. We consider both settings where panel data are available and settings where only repeated cross-section data are available. We contribute to the DID literature in different fronts. First, we derive doubly robust (DR) estimands for the ATT under DID settings and propose DR DID estimators for the ATT that are consistent when either a working (parametric) model for the propensity score or a working (parametric) model for the outcome evolution for the comparison group is correctly specified. The setting where only repeated cross-section data are available is particularly interesting. We propose two different DR DID estimators for the ATT that differ from each other depending on whether or not one models the outcome regression for the treated group in both pre and post-treatment periods. Nonetheless, we show DR property does not depend on such a choice.

Second, we derive the semiparametric efficiency bounds for the ATT under DID designs. The semiparametric efficiency bounds we derive are nonparametric in the sense that we do not assume researchers have additional knowledge about outcome regressions or the propensity score functional forms. As so, these bounds provide a standard against which one can compare the efficiency of any (regular) semiparametric DID estimator for the ATT. Here, it is also worth stressing that these semiparametric efficiency bounds explicitly incorporate all the restrictions implied by the invoked identification assumptions. Importantly, these restrictions differ depending on whether panel or repeated cross-section data are available. In both cases they involve the moment restrictions implied by the conditional PTA, though, when repeated cross-section data are available, they also include the restrictions implied by the identifying assumption that the joint distribution of covariates and treatment status is invariant to the sampling period (pre and post-treatment). We emphasize that failing to account for all these implied restrictions can lead to discrepancies on the derived efficiency bound, which, in turn, may suggest that some estimator is semiparametrically efficient when, in fact, it is not.

With the semiparametric efficiency bounds at hand, we can answer several questions that one may have. For instance, one may wonder whether there are efficiency gains associated with having access to panel instead of repeated cross-section data. By directly comparing the efficiency bounds under these two setups, we not only show that the answer to the aforementioned question is yes, but also show that such gains tend to be larger when the sample sizes of the pre and post-treatment repeated cross-section data are more imbalanced.

Another natural question that arises is whether our proposed DR DID estimators can attain the semiparametric efficiency bound. We show that when the working models for the propensity score and for the outcome evolution for the comparison group are correctly specified, our proposed DR DID estimator for the panel data setup is locally efficient, though the DR DID estimators for the cross-section setup are not. In fact, when only repeated cross-section data are available, we show that our proposed DR DID estimator that relies on modelling the propensity score and the outcome evaluation of both the treated and comparison groups attains the semiparametric efficiency bound when all working models are correctly specified. We quantify the loss of efficiency associated with using the inefficient DR DID estimator instead of the locally efficient one, and illustrate via Monte Carlo simulations that such a loss can indeed be large.

Our proposed methodology accommodates linear and nonlinear working models for the nuisance functions. We establish $\sqrt{n}$ -consistency and asymptotic normality of the proposed DR DID estimators when generic parametric working models are used for the nuisance functions. In doing so we emphasize that, in general, the DR property of our estimators is with respect to consistency and not to inference. In other words, the exact form of the asymptotic variance of our proposed estimators depends on whether the propensity score and/or the outcome regression models are correctly specified. Given that, in practice, one does not know a priori which models are correctly specified, one should consider the estimation effects from all first-step estimators when estimating the asymptotic variance. Failing to do so may lead to invalid inference procedures.

Motivated by this observation, a third contribution of this paper is to show that, by paying particular attention to the estimation method used for estimating the nuisance parameters, it is sometimes possible to construct computationally simple DID estimators for the ATT that are not only DR consistent and locally semiparametric efficient, but are also doubly robust for inference. These further improved DR DID estimators are particularly attractive and easy to implement when researchers are comfortable with a logistic working model for the propensity score and with linear regression working models for the outcome of interest.

Related literature: Our proposal builds on two branches of the causal inference literature. First, our methodological results are intrinsically related to other DID papers; for an overview, see e.g., Section 6.5 of Imbens and Wooldridge (2009) and references therein. Two leading contributions in this branch of literature that are particularly relevant to this paper are Heckman et al. (1997), who propose kernel-based DID regression estimators, and Abadie (2005), who proposes (parametric and nonparametric) DID inverse probability weighted (IPW) estimators. We note that when the dimension of available covariates is high or even moderate, fully nonparametric procedures usually do not lead to informative inference because of the “curse of dimensionality”. In these cases, researchers often adopt parametric methods. Our DR DID estimators fall in this latter category.

Second, our results are also directly related to the literature on doubly robust estimators, see Robins et al. (1994), Scharfstein et al. (1999), Bang and Robins (2005), Wooldridge (2007), Chen et al. (2008), Cattaneo (2010), Graham et al. (2012, 2016), Vermeulen and Vansteelandt (2015), Lee et al. (2017), Słoczyński and Wooldridge (2018), Rothe and Firpo (2018), Muris (2019), among many others; for an overview, see section 2 of Słoczyński and Wooldridge (2018), and Seaman and Vansteelandt (2018). Recently, DR estimators have also been playing an important role when one uses data-adaptive, “machine learning” estimators for the nuisance functions, see e.g., Belloni et al. (2014), Farrell (2015), Chernozhukov et al. (2017), Belloni et al. (2017), and Tan (2019). As so, these papers are also broadly related to our proposal, even though we use parametric first-step estimators. On the other hand, we note that the aforementioned papers focus on either the “selection on observables” or “IV/LATE” type assumptions, whereas we pay particular attention to the conditional DID design. Thus, our results complement theirs.

To derive the semiparametric efficiency bounds for the ATT under the DID framework, we build on Hahn (1998) and Chen et al. (2008). Although we follow the structure of semiparametric efficiency bound derivation of the aforementioned papers (which, in turn, follow Newey (1990)), our derived semiparametric efficiency bounds complement theirs as we focus on DID designs while Hahn (1998) and Chen et al. (2008) results rely on “selection on observables” type assumptions in cross-section setups.

Our results for the further improved DR DID estimators build on Vermeulen and Vansteelandt (2015), who propose estimators that are DR for inference in cross-section setups under selection on observables type assumptions. We extend Vermeulen and Vansteelandt (2015) proposal to DID settings with both panel and repeated cross-section data. Our further improved DR DID estimators also build on Graham et al. (2012), as their proposed propensity score estimator is one important component of our proposal.

Finally, in work related but independent from ours, Zimmert (2019) provides high-level conditions under which one can use “machine-learning” first-step estimators when estimating the ATT in DID setups. His results complement ours, though we note that his proposed estimators for the repeated cross-section case do not attain the semiparametric efficiency bound derived in this paper, and the loss of efficiency can be of first-order importance. We also note that Zimmert (2019) does not provide a detailed comparison between the panel and repeated cross-section data setups like we do, nor discusses DR inference procedures, which are particularly relevant under model misspecifications.

Organization of the paper: In the next section, we describe this paper’s framework, briefly give an overview of the existing DID estimators and describe how we combine the strengths of each method to form our DR DID estimands. We also derive semiparametric efficiency bounds for the ATT in Section 2. In Section 3, we propose different DR DID estimators, derive their large sample properties, and show that we can get improved DR DID estimators by paying particular attention to the estimation method used for estimating the nuisance parameters. We examine the finite sample properties of our proposed methodology by means of a Monte Carlo study in Section 4, and provide an empirical illustration in Section 5. Section 6 concludes. Mathematical proofs are gathered in the Supplemental Appendix.111The Supplemental Appendix is available at https://pedrohcgs.github.io/files/DR-DIDAppendix.pdf Finally, all proposed policy evaluation tools discussed in this article can be implemented via the open-source R package DRDID, which is freely available from GitHub (https://github.com/pedrohcgs/DRDID).

2 Difference-in-differences

2.1 Background

We first introduce the notation we use throughout the article. We focus on the case where there are two treatment periods and two treatment groups. Let $Y_{it}$ be the outcome of interest for unit $i$ at time $t$ . We assume that researchers have access to outcome data in a pre-treatment period $t=0$ and in a post-treatment period $t=1$ . Let $D_{it}=1$ if unit $i$ is treated before time $t$ and $D_{it}=0$ otherwise. Note that $D_{i0}=0$ for every $i$ , allowing us to write $D_{i}=D_{i1}$ . Using the potential outcome notation, denote $Y_{it}\left(0\right)$ the outcome of unit $i$ at time $t$ if it does not receive treatment by time $t$ and $Y_{it}\left(1\right)$ the outcome for the same unit if it receives treatment. Thus, the realized outcome for unit $i$ at time $t$ is $Y_{it}=D_{i}Y_{it}\left(1\right)+\left(1-D_{i}\right)Y_{it}\left(0\right)$ . A vector of pre-treatment covariates $X_{i}$ is also available. Henceforth, we assume that the first element of $X_{i}$ is a constant.

In the rest of the article, we assume that either panel or repeated cross-section data on $\left(Y_{it},D_{i},X_{i}\right)$ , $t=0,1$ are available. When repeated cross-section data are available, we follow Abadie (2005) and assume that covariates and treatment status are stationary. We formalize these conditions in the following assumption. Let $T_{i}$ be a dummy variable that takes value one if the observation $i$ is only observed in the post-treatment period, and zero if observation $i$ is only observed in the pre-treatment period. Define $Y_{i}=T_{i}Y_{i1}+\left(1-T_{i}\right)Y_{i0}$ , and let $n_{1}$ and $n_{0}$ be the sample sizes of the post-treatment and pre-treatment periods such that $n=n_{1}+n_{0}$ . Finally, let $\lambda=\mathbb{P}\left(T=1\right)\in\left(0,1\right)$ .

Assumption 1.

Assume that either $\left(a\right)$ the data $\{Y_{i0},Y_{i1},D_{i},X_{i}\}_{i=1}^{n}$ are independent and identically distributed $(iid)$ ; or $\left(b\right)$ the pooled repeated cross-section data $\left\{Y_{i},D_{i},X_{i},T_{i}\right\}_{i=1}^{n}$ consist of $iid$ draws from the mixture distribution

[TABLE]

*where $\left(y,d,x,t\right)\in\mathbb{R\times}\left\{0,1\right\}\times\mathbb{R}^{k}\times\left\{0,1\right\}$ , with the joint distribution of $\left(D,X\right)$ being invariant to $T$ . *

Assumption 1 $(a)$ covers the case where panel data are available, whereas Assumption 1 $(b)$ covers the case where repeated cross-section data are available, and allows for different sampling schemes. For instance, it accommodates the binomial sampling scheme where an observation $i$ is randomly drawn from either $\left(Y_{1},D,X\right)$ or $\left(Y_{0},D,X\right)$ with fixed probability $\lambda$ (here, $T$ is a non-degenerated random variable). It also accommodates the “conditional” sampling scheme where $n_{1}$ observations are sampled from $\left(Y_{1},D,X\right)$ , $n_{0}$ observations are sampled from $\left(Y_{0},D,X\right)$ and $\lambda=n_{1}/n$ (here, $T$ is treated as fixed). On the other hand, Assumption 1 $\left(b\right)$ rules out settings with compositional changes in $\left(D,X\right)$ , see e.g. Hong (2013) for a discussion.

The parameter of interest is the average treatment effect on the treated,

[TABLE]

As expectations are linear operators and $Y_{i1}\left(1\right)=Y_{i1}$ if $D_{i}=1$ , we can rewrite the ATT as222Throughout the rest of the paper, to ease the notation burden we denote $\mathbb{E}\left[\cdot\right]$ as generic expectations. In the case of panel data, such expectations are with respect to the distribution of $\left(Y_{0},Y_{1},D,X\right)$ . In the case of repeated cross-section data, the expectations are with respect to the mixture distribution $\sum_{t=0}^{1}\mathbb{P}\left(T=t\right)\cdot\mathbb{P}\left(Y_{t}\leq y,D=d,X\leq x|T=t\right)$ .

[TABLE]

where we drop subscript $i$ to ease notation; we follow this convention throughout the paper. From the above representation, it is clear that the main challenge in identifying the ATT is to compute $\mathbb{E}[Y_{i1}(0)|D_{i}=1]$ from the observed data. To overcome this challenge, we invoke the following assumptions.

Assumption 2.

$\mathbb{E}[Y_{1}(0)-Y_{0}(0)|D=1,X]=\mathbb{E}[Y_{1}(0)-Y_{0}(0)|D=0,X]$ * almost surely (a.s.). *

Assumption 3.

*For some $\varepsilon>0$ , $\mathbb{P}\left(D=1\right)>\varepsilon$ and $\mathbb{P}\left(D=1|X\right)\leq 1-\varepsilon$ a.s.. *

Assumption 2, which we refer to as the conditional PTA throughout the paper, states that in the absence of treatment, the average conditional outcome of the treated and the comparison groups would have evolved in parallel. Note that Assumption 2 allows for covariate-specific time trends, though it rules out unit specific trends. Assumption 3 is an overlap condition and states that at least a small fraction of the population is treated and that for every value of the covariates $X$ , there is at least a small probability that the unit is not treated. These two assumptions are standard in conditional DID methods, see e.g. Heckman et al. (1997), Heckman et al. (1998), Blundell et al. (2004), Abadie (2005) and Bonhomme and Sauder (2011).

Under Assumptions 1-3, there are two main flexible estimation procedures to estimate the ATT: the outcome regression (OR) approach, see e.g. Heckman et al. (1997), and the IPW approach, see e.g. Abadie (2005). The OR approach relies on researchers ability to model the outcome evolution. In such cases, under the aforementioned assumptions one can estimate the ATT using

[TABLE]

where $\bar{Y}_{d,t}=\sum_{i\left|D_{i}=d,T_{i}=t\right.}Y_{it}/n_{d,t}$ is the sample average outcome among units in treatment group $d$ and time $t$ , and $\widehat{\mu}_{d,t}\left(x\right)$ is an estimator of the true, unknown $m_{d,t}\left(x\right)\equiv\mathbb{E}[Y_{t}|D=d,X=x]$ ,333In the repeated cross-section case, $m_{d,t}\left(x\right)=\mathbb{E}\left[Y|D=d,T=t,X=x\right]$ . In the next section, we differentiate the notation for the panel data and repeated cross-section case to avoid potential confusions. see e.g. Heckman et al. (1997).

The IPW approach proposed by Abadie (2005) avoids directly modelling the outcome evolution and exploits that, under Assumptions 1-3, the ATT can be expressed as

[TABLE]

when panel data are available, and as

[TABLE]

when repeated cross-section data are available, where $p\left(X\right)\equiv\mathbb{P}\left(D=1|X\right)$ is the true, unknown propensity score. Abadie’s identification results suggest simple two-step estimators for the ATT that do not involve outcome regressions. For instance, when panel data are available, Abadie (2005) proposes the following Horvitz and Thompson (1952) type IPW estimator,

[TABLE]

where $\widehat{\pi}\left(x\right)$ is an estimator of the true, unknown $p\left(x\right)$ , and for a generic random variable $Z$ , $\mathbb{E}_{n}\left[Z\right]=n^{-1}\sum_{i=1}^{n}Z_{i};$ the estimator for the repeated cross-section case is formed using the analogous procedure.

It is important to emphasize that the reliability of ATT estimators based on the OR and the IPW approaches depends on different, non-nested conditions. For the OR approach, the consistency of the ATT estimator (2.2) relies on the estimators of $m_{d,t}\left(\cdot\right)$ , $\widehat{\mu}_{d,t}\left(\cdot\right)$ , being correctly specified, whereas the IPW estimator (2.4) relies on the propensity score estimator $\widehat{\pi}(\cdot)$ of $p\left(\cdot\right)$ being correctly specified. As so, in practice, it may be hard to “rank” these two approaches in terms of their robustness to model misspecification.

Remark 1.

It is common to see practitioners adopting the two-way fixed effects linear regression model

[TABLE]

and interpreting estimates of $\tau^{fe}$ as estimates of the ATT, see e.g. chapter 5.2 in Angrist and Pischke (2009). Although (2.5) may be perceived as a “natural” specification, it implicitly imposes additional restrictions on the data generating process beyond Assumptions 1-3. More specifically, (2.5) implicitly imposes that $\left(i\right)\mathbb{E}\left[Y_{1}\left(1\right)-Y_{1}\left(0\right)|X,D=1\right]=\tau^{fe}\leavevmode\nobreak\ a.s.$ , i.e., it assumes homogeneous (in $X$ ) treatment effects, and $\left(ii\right)$ for $d=0,1$ , $\mathbb{E}\left[Y_{1}-Y_{0}|X,D=d\right]=\mathbb{E}\left[Y_{1}-Y_{0}|D=d\right]\leavevmode\nobreak\ a.s.$ , i.e., it rules out $X$ -specific trends in both treated and comparison groups.444Note that under Assumptions 1-3, (2.5) suggests that, with probability one, $\mathbb{E}\left[Y_{1}\left(1\right)|X,D=1\right]=\alpha_{1}+\alpha_{2}+\alpha_{3}+\tau+\theta^{\prime}X\leavevmode\nobreak\ ,$ and $\mathbb{E}\left[Y_{1}\left(0\right)|X,D=1\right]=\mathbb{E}[Y_{0}|D=1,X]+\left(\mathbb{E}[Y_{1}|D=0,X]-\mathbb{E}[Y_{0}|D=0,X]\right)=\alpha_{1}+\alpha_{2}+\alpha_{3}+\theta^{\prime}X\leavevmode\nobreak\$ . Point $\left(i\right)$ now follows directly. Point $\left(ii\right)$ follows from analogous arguments. When these additional restrictions are not satisfied, the estimand $\tau^{fe}$ is, in general, different from the ATT, and policy evaluation based on it may be misleading. We further illustrate this point using Monte Carlo simulations in Section 4; see also Słoczyński (2018) for related results.

2.2 Doubly robust difference-in-differences estimands

In this section, we argue that instead of choosing between the OR and the IPW approaches, one can combine them to form doubly robust (DR) moments/estimands for the ATT. Here, double robustness means that the resulting estimand identifies the ATT even if either (but not both) the propensity score model or the outcome regression models are misspecified. As so, the DR DID estimand for the ATT shares the strengths of each individual DID method and, at the same time, avoids some of their weaknesses.

Before describing how we exactly combine the OR and the IPW approaches to form our DR DID estimand, we need to introduce some additional notation. Let $\pi\left(X\right)$ be an arbitrary model for the true, unknown propensity score. When panel data are available, let $\Delta Y=Y_{1}-Y_{0}$ and define $\mu_{d,\Delta}^{p}\left(X\right)\equiv\mu_{d,1}^{p}\left(X\right)-\mu_{d,0}^{p}\left(X\right)$ , $\mu_{d,t}^{p}\left(x\right)$ being a model for the true, unknown outcome regression $m_{d,t}^{p}\left(x\right)\equiv\mathbb{E}[Y_{t}|D=d,X=x]$ , $d,t=0,1$ . When only repeated cross-section data are available, let $\mu_{d,t}^{rc}\left(x\right)$ be an arbitrary model for the true, unknown regression $m_{d,t}^{rc}\left(x\right)\equiv\mathbb{E}[Y|D=d,T=t,X=x]$ , $d,t=0,1$ , and for, $d=0,1,$ $\mu_{d,Y}^{rc}\left(T,X\right)\equiv T\cdot\mu_{d,1}^{rc}\left(X\right)+\left(1-T\right)\cdot\mu_{d,0}^{rc}\left(X\right)$ , and $\mu_{d,\Delta}^{rc}\left(X\right)\equiv\mu_{d,1}^{rc}\left(X\right)-\mu_{d,0}^{rc}\left(X\right)$ .

For the case in which panel data are available, we consider the estimand

[TABLE]

where, for a generic $g$ ,

[TABLE]

For the repeated cross-section case, we consider two different estimands,

[TABLE]

and

[TABLE]

where, for a generic $g$ ,

[TABLE]

and, for $t=0,1$ ,

[TABLE]

Theorem 1.

Let Assumptions 1-3 hold. Then:

$\left(a\right)\leavevmode\nobreak\$ When panel data are available, $\tau^{dr,p}=\tau$ if either (but not necessarily both) $\pi(X)=p\left(X\right)$ $a.s.$ or $\mu_{\Delta}^{p}\left(X\right)=m_{0,1}^{p}\left(X\right)-m_{0,0}^{p}\left(X\right)$ $a.s.;$

$\left(b\right)\leavevmode\nobreak\$ *When repeated cross-section data are available, $\tau_{1}^{dr,rc}=\tau_{2}^{dr,rc}=\tau$ if either (but not necessarily both) $\pi(X)=p\left(X\right)$ $a.s.$ or $\mu_{0,\Delta}^{rc}\left(X\right)=m_{0,1}^{rc}\left(X\right)-m_{0,0}^{rc}\left(X\right)$ $a.s..$ *

Theorem 1 states that provided that at least one of the working nuisance models is correctly specified, we can recover the ATT with either panel or repeated cross-section data. Thus, our proposed DR DID estimands are “less demanding” in terms of the researchers’ ability to correctly specify models for the nuisance functions than either the OR or the IPW approach.

Given that we consider two different estimands for the case of repeated cross-section, it is interesting to use Theorem 1 to compare them. Given that $\tau_{1}^{dr,rc}$ does not rely on OR models for the treated group but $\tau_{2}^{dr,rc}$ does, one could a priori expect that $\tau_{1}^{dr,rc}$ would be more robust against model misspecification than $\tau_{2}^{dr,rc}$ . Nonetheless, Theorem 1 states that this is not the case as they identify the ATT under the same conditions. At this stage, one may wonder how this is possible. To answer such a query, it suffices to remember that, under the stationarity condition in Assumption 1 $\left(b\right)$ , for any generic integrable and measurable function $g$ , $\mathbb{E}\left[\left.g\left(X\right)\right|D=1\right]=\mathbb{E}\left[\left.g\left(X\right)\right|D=1,T=t\right]$ , $t=0,1$ . Given that this holds for any generic function $g$ , it must also hold for $\mu_{1,t}^{rc}\left(\cdot\right)-\mu_{0,t}^{rc}\left(\cdot\right),$ $t=0,1$ , even when $\mu_{d,t}^{rc}\left(\cdot\right)$ are misspecified models of $m_{d,t}^{rc}\left(\cdot\right)$ . Such a result reveals that modeling the OR for the treat group can be “harmless” in terms of identification, provided that these additional models are incorporated into $\tau_{1}^{dr,rc}$ in an appropriate manner.

2.3 Semiparametric efficiency bound

In the previous subsection, we derived DR moment equations for the ATT under the DID framework and showed that the resulting estimands are more robust against model misspecifications than DID estimands based on either the OR or the IPW approach. In this subsection, we shift our attention from “robustness” to efficiency. More precisely, we calculate the semiparametric efficiency bound for the ATT under Assumptions 1-3 when either panel or repeated cross-section data are available. These results provide the semiparametric analog of the Cramér–Rao lower bound commonly used in fully parametric procedures. As so, they provide a benchmark that researchers can use to assess whether any given (regular) semiparametric DID estimator for the ATT is fully exploiting the empirical content of Assumptions 1-3.

Let $m_{0,\Delta}^{p}\left(x\right)\equiv m_{0,1}^{p}\left(x\right)-m_{0,0}^{p}\left(x\right)$ , and, for $d=0,1,$ $m_{d,\Delta}^{rc}\left(X\right)\equiv m_{d,1}^{rc}\left(X\right)-m_{d,0}^{rc}\left(X\right)$ . Recall that $\lambda\equiv\mathbb{P}\left(T=1\right)$ . Next proposition displays the semiparametric efficiency bound for the ATT when one has access to panel data and when one has access to repeated cross-section data. To simplify exposition, we abstract from additional technical discussions related to the conditions to guarantee quadratic mean differentiability and their implications for the precise definition of efficient influence function ; see, e.g., Chapter 3 of Bickel et al. (1998) for additional details.

Proposition 1.

Let Assumptions 1-3 hold. Then:

$\left(a\right)$ * When panel data are available, the efficient influence function for the $ATT$ is*

[TABLE]

and the semiparametric efficiency bound for all regular estimators for the ATT is

[TABLE]

$\left(b\right)$ * When only repeated cross-section data are available, the efficient influence function for the $ATT$ is*

[TABLE]

and the semiparametric efficiency bound for all regular estimators for the ATT is

[TABLE]

It is interesting to compare $\eta^{e,p}\left(D,X\right)$ with $\eta^{e,rc}\left(D,T,X\right)$ . First, note that the first component of their efficient influence functions are analogous to each other, and depends on the true, unknown conditional ATT, $m_{1,\Delta}\left(X\right)-m_{0,\Delta}\left(X\right)$ .555To avoid excessive notational burden, we supress the “ $p$ ” and “ $rc$ ” superscripts unless their omission leads to confusion. The second and third terms in (2.11) and (2.13) are more different from each other. For $\eta^{e,p}$ , the availability of panel data implies that $Y_{1}$ and $Y_{0}$ are observed for all units, and, therefore, we can directly reweight $\Delta Y-m_{1,\Delta}\left(X\right)$ and $\Delta Y-m_{0,\Delta}\left(X\right)$ . In contrast, when only repeated cross-section data are available, one observes $Y_{t}$ only if $T=t$ , $t=0,1$ , and, therefore, the efficient influence function (2.13) depends on different weights for each pair $\left(D,T\right)$ $\in\left\{0,1\right\}^{2}$ . In this latter case, we also stress the importance of imposing the stationarity condition in Assumption 1(b) when deriving the efficient influence function (2.13) – failing to do so will suggest an “efficiency bound” that is wider than (2.14).

It is also worth mentioning that the efficient influence functions (2.11) and (2.13) depend on the true, unknown, outcome regression functions for the treated group, $m_{1,1}\left(\cdot\right)$ and $m_{1,0}\left(\cdot\right)$ , in an asymmetric manner. On one hand, when panel data are available, by simple manipulation, we can rewrite $\eta^{e,p}$ as

[TABLE]

emphasizing that the efficient influence function for the ATT when panel data are available does not depend on $m_{1,1}\left(\cdot\right)$ and $m_{1,0}\left(\cdot\right)$ . This is in sharp contrast to the case where only repeated cross-section data are available.

Another interesting question raised by Proposition 1 is whether the semiparametric efficiency bound for the case of repeated cross-section data is larger than the one for the case of panel data. In order to answer this question, we consider the case where $T$ is independent of $\left(Y_{1},Y_{0},D,X\right)$ , so that Assumptions 1 $\left(a\right)$ and 1 $\left(b\right)$ are compatible with each other.666This “restriction” does not affect the semiparametric efficiency bound for the case where only repeated cross-section data are available, as it does not impose additional restrictions on the observed data.

Corollary 1.

Let Assumptions 1-3 hold, and assume that $T$ is independent of $\left(Y_{1},Y_{0},D,X\right)$ . Then,

[TABLE]

In other words, under the DID framework it is possible to form more efficient estimators for the ATT when panel data are available than when only repeated cross-section data are available. In addition, from Corollary 1, we can also see that the efficiency loss is convex in $\lambda\,$ , implying that the loss of efficiency is bigger when the pre and post-treatment sample sizes are more imbalanced. In fact, when

[TABLE]

we can show that $\lambda=0.5$ is optimal . However, when (2.15) does not hold, the optimal $\lambda$ depends on the data in a more complicated manner, and is given by $\lambda=\left.\tilde{\sigma}_{1}\right/\left(\tilde{\sigma}_{0}+\tilde{\sigma}_{1}\right)$ , where, for $t=0,1$

[TABLE]

These results suggest that, in principle, one may benefit from “oversampling” from either the pre or post-treatment period. However, it is, in general, not feasible to know the optimal $\lambda$ during the design stage, i.e., at the pre-treatment period, since $\tilde{\sigma}_{1}^{2}$ depends on the outcome data from the post-treatment period. Thus, if one were to design the DID study with repeated cross-section units, it seems that setting $\lambda=0.5$ would be a “reasonable” choice.

3 Estimation and inference

In this section, we build on the DR DID estimands in Theorem 1 and the semiparametric efficiency bounds in Proposition 1, and discuss estimation and inference procedures for the ATT in DID designs. Indeed, the moment equations (2.6), (2.8), and (2.9) suggest a simple two-step strategy to estimate the ATT. In the first step, one estimates the true, unknown $p\left(\cdot\right)$ using $\pi(\cdot),$ and the true, unknown $m_{d,t}^{p}\left(\cdot\right)$ ( $m_{d,t}^{rc}\left(\cdot\right)$ ) using $\mu_{d,t}^{p}\left(\cdot\right)$ ( $\mu_{d,t}^{rc}\left(\cdot\right)$ ), $d,t=0,1$ , when panel data (repeated cross-section data) are available. In the second step, one plugs the fitted values of the estimated propensity score and regression models into the sample analogue of $\tau^{dr,p},$ $\tau_{1}^{dr,rc}$ , or $\tau_{2}^{dr,rc}$ .

Although, in principle, one can use semi/non-parametric estimators for both the outcome regressions and the propensity score, see e.g. Heckman et al. (1997), Abadie (2005), Chen et al. (2008) and Rothe and Firpo (2018), in what follows ,we focus our attention on generic parametric first-step estimators. More precisely, we assume that $\pi\left(x;\gamma^{\ast}\right)$ is a parametric model for $p\left(x\right),$ such that $\pi$ is known up to the finite dimensional pseudo-true parameter $\gamma^{\ast}$ . Analogously, for $d,t=0,1$ , $\mu_{d,t}^{p}\left(x;\beta_{d,t}^{\ast,p}\right)$ (and $\mu_{d,t}^{rc}\left(x;\beta_{d,t}^{\ast,rc}\right)$ ) is a parametric model for $m_{d,t}^{p}\left(x\right)$ ( $m_{d,t}^{rc}\left(x\right)$ ), such that $\mu_{d,t}^{p}$ ( $\mu_{d,t}^{rc}$ ) is known up to the finite dimensional pseudo-true parameter $\beta_{d,t}^{\ast,p}$ ( $\beta_{d,t}^{\ast,rc}$ ). This is perhaps the most popular approach adopted by practitioners, particularly when the available sample size is moderate and/or the dimension of available covariates is high or even moderate, as the “curse of dimensionality” usually prevents one to adopt fully nonparametric procedures.777Let $g\left(x\right)$ be a generic notation for $p\left(x\right)$ , $m_{d,t}^{l}\left(X\right),$ $m_{d,t}^{l}\left(X\right)$ , $d,t=0,1$ , $l=p,rc.$ From Newey (1994), Chen et al. (2003), Ai and Chen (2003, 2007, 2012), and Chen et al. (2008), one can see that the use of nonparametric first-step estimators $\widehat{g}\left(x\right)$ of $g\left(x\right)$ is warranted provided that $\left\|\widehat{g}\left(x\right)-g\left(x\right)\right\|_{\mathcal{H}}=o_{p}\left(n^{-1/4}\right)$ for a pseudo-metric $\left\|\cdot\right\|_{\mathcal{H}}$ , $\mathcal{H}$ being a vector space of functions. However, when the dimension of $X$ is moderate or large, as is usually the case in many empirical applications, conditons ensuring that $\left\|\widehat{g}\left(x\right)-g\left(x\right)\right\|_{\mathcal{H}}=o_{p}\left(n^{-1/4}\right)$ can be rather stringent because of the “curse of dimensionality”.

In the case when panel data are available, our proposed DR DID estimator for the ATT is based on (2.6) and is given by

[TABLE]

where

[TABLE]

$\widehat{\gamma}$ is an estimator for the pseudo-true $\gamma^{\ast}$ , $\widehat{\beta}_{0,t}^{p}$ is an estimator for pseudo-true $\beta_{0,t}^{\ast,p}$ , $t=0,1$ , and for a generic $\beta_{0}$ and $\beta_{1}$ , $\mu_{0,\Delta}^{p}(\cdot;\beta_{0},\beta_{1})=\mu_{0,1}^{p}\left(\cdot;\beta_{1}\right)-\mu_{0,0}^{p}\left(\cdot;\beta_{0}\right)$ .

When only repeated cross-section data are available, we propose two different DR DID estimators for the ATT. The first one, which is based on (2.8) and can be interpreted as the analogue of $\widehat{\tau}^{dr,p}$ , is given by

[TABLE]

where $\mu_{0,Y}^{rc}\left(T,\cdot;\beta_{0,0}^{rc},\beta_{0,1}^{rc}\right)=T\cdot\mu_{0,1}^{rc}\left(\cdot;\beta_{0,1}^{rc}\right)+\left(1-T\right)\cdot\mu_{0,0}^{rc}\left(\cdot;\beta_{0,0}^{rc}\right)$ , $\widehat{\beta}_{d,t}^{rc}$ is an estimator for the pseudo-true $\beta_{d,t}^{\ast,rc}$ , $d,t=0,1$ , and the weights $\widehat{w}_{1}^{rc}\left(D,T\right)$ and $\widehat{w}_{0}^{rc}\left(D,T,X;\widehat{\gamma}\right)$ are, respectively, defined as the sample analogues of $w_{1}^{rc}\left(D,T\right)$ and $w_{0}^{rc}\left(D,T,X;g\right)$ defined in (2.10), but with $\pi\left(x;\widehat{\gamma}\right)$ playing the role of $g$ .

The second DR DID estimator for the case of repeated cross-section builds on (2.9) and is given by

[TABLE]

where $\mu_{d,\Delta}^{rc}\left(\cdot;\beta_{d,1}^{rc},\beta_{d,0}^{rc}\right)=\mu_{d,1}^{rc}\left(\cdot;\beta_{d,1}^{rc}\right)-\mu_{d,0}^{rc}\left(\cdot;\beta_{d,0}^{rc}\right)$ , and the weights $\widehat{w}_{1,t}^{rc}\left(D,T\right)$ and $\widehat{w}_{0,t}^{rc}\left(D,T,X;\widehat{\gamma}\right)$ are, respectively, defined as the sample analogues of $w_{1,t}^{rc}\left(D,T\right)$ and $w_{0,t}^{rc}\left(D,T,X;g\right)$ , $t=0,1$ , defined below (2.10), but with $\pi\left(x;\widehat{\gamma}\right)$ playing the role of $g$ .

As we show in the Appendix A, it is relatively straightforward to derive the asymptotic properties of $\widehat{\tau}^{dr,p}$ , $\widehat{\tau}_{1}^{dr,rc}$ and $\widehat{\tau}_{2}^{dr,rc}$ using generic first-step estimators that satisfy some relatively weak, high-level conditions; see Theorems A.1 and A.2 in Appendix A. Indeed, Theorem A.1 indicates that $\widehat{\tau}^{dr,p}$ is doubly robust, and also locally semiparametrically efficient, i.e., its asymptotic variance achieves the semiparametric efficiency bound when the working models for the nuisance functions are correctly specified. Theorem A.2 also indicates that both $\widehat{\tau}_{1}^{dr,rc}$ and $\widehat{\tau}_{2}^{dr,rc}$ are doubly robust when repeated cross-section data are available. However, Theorem A.2 also highlights that $\widehat{\tau}_{2}^{dr,rc}$ is locally semiparametrically efficient, whereas $\widehat{\tau}_{1}^{dr,rc}$ is not. In other words, when repeated cross-section data are available, $\widehat{\tau}_{2}^{dr,rc}$ tends to have more attractive properties than $\widehat{\tau}_{1}^{dr,rc}$ , regardless of the first-step estimators used.

Although the results in Theorems A.1 and A.2 accommodate a variety of different first-step estimators, in practice, one still needs to choose a particular estimation procedure to be implemented. In what follows, we attempt to provide some guidance on the choice of first-step estimators with the goal of further improving the (generic) DR DID estimators. We are particularly interested in forming DR DID estimators that are not only doubly robust in terms of consistency—like described above—but also doubly robust for inference, i.e., their asymptotic linear representation is also doubly robust. The attractiveness of forming estimators that are DR for inference is that there is no estimation effect from first-step estimators, which, in turn, implies that the asymptotic variance of the results DR DID estimator for the ATT is invariant to which working models for the nuisance functions are correctly specified. In practice, this usually translates to simpler and more stable inference procedures.

To derive these improved DR DID estimators, we focus on the case where a researcher is comfortable with linear regression working models for the outcome of interest, a logistic working model for the propensity score, and with covariates $X$ entering all the nuisance models in a symmetric manner. Although these modelling conditions are more stringent than those allowed by our generic DR DID estimators discussed in Appendix A, they are much weaker than those implicitly imposed in the TWFE specification (2.5), and can be seen as the default choice in many applications. Hence, these extra assumptions can be seen as a reasonable compromise to get further improved DR DID estimators that are also computationally tractable and easy to implement in practice.

3.1 Improved DR DID estimators when panel data are available

As discussed above, we consider the following working models for the nuisance functions:

[TABLE]

Our proposed improved DR DID estimator is given by the three-step estimator

[TABLE]

where the first two-steps consist of computing

[TABLE]

while in the third and last step, one plugs the fitted values of the working models (3.5) into the sample analogue of $\tau^{dr,p}$ . Here, note that $\widehat{\gamma}^{ipt}$ is the inverse probability tilting estimator proposed by Graham et al. (2012) in a different context, while $\widehat{\beta}_{0,\Delta}^{wls,p}$ is simply the weighted least squares estimator for $\beta_{0,\Delta}^{\ast,p}$ .

At this point, one may wonder why we use the estimators $\widehat{\gamma}^{ipt}$ and $\widehat{\beta}_{0,\Delta}^{wls,p}$ instead of other available alternatives. To answer such a query, recall that the main goal here is to propose DID estimators for the ATT that are not only DR consistent but also DR for inference, i.e., the exact form of their asymptotic variance does not depend on which working models for the nuisance functions are correctly specified. As it turns out, the key to obtain DID estimators for the ATT that are also DR for inference is to choose first-step estimators for the nuisance parameters, say $\widehat{\gamma}$ and $\widehat{\beta}$ , such that the limiting distribution of the resulting DR DID estimator $\widehat{\tau}^{dr,p}$ is equivalent to that of the infeasible DR DID estimator that uses the pseudo-true values of $\widehat{\gamma}$ and $\widehat{\beta}$ , say $\gamma^{\ast}$ and $\beta^{\ast}$ . In a more precise manner, in order to get DID estimators that are DR for inference, we need to guarantee that there will be no estimation effect from the first stage.

In Appendix A, we show that the estimation effect associated with using generic first-step estimators $\widehat{\gamma}$ and $\widehat{\beta}$ is given by $\eta_{est}^{p}\left(W;\gamma^{\ast},\beta^{\ast}\right)$ as defined in (A.2). By paying closer attention to the exact form of $\eta_{est}^{p}\left(W;\gamma^{\ast},\beta^{\ast}\right)$ , one can see that if

[TABLE]

then there will be no estimation effect from the first stage. As the first component of $X$ is assumed to be constant and we adopt the working models (3.5), it follows that (3.6) reduces to

[TABLE]

However, as $n\rightarrow\infty$ , these two vectors of moment conditions follow from the first-order conditions of the optimization problems associated with $\widehat{\gamma}^{ipt}$ and $\widehat{\beta}_{0,\Delta}^{wls,p}$ , respectively, even when these working models are misspecified. Hence, by using $\widehat{\gamma}^{ipt}$ and $\widehat{\beta}_{0,\Delta}^{wls,p}$ , we guarantee that $\widehat{\tau}_{imp}^{dr,p}$ is doubly robust for inference as there is no estimation effect from replacing the pseudo-true parameters $\gamma^{\ast,ipt}$ and $\beta_{0,\Delta}^{\ast,wls,p}$ with their estimators $\widehat{\gamma}^{ipt}$ and $\widehat{\beta}_{0,\Delta}^{wls,p}$ , respectively.

The next theorem formalizes this discussion. Define

[TABLE]

and let

[TABLE]

Theorem 2.

Suppose Assumptions 1-3 and Assumptions A.1-A.2 stated in Appendix A hold, and that the working nuisance models (3.5) are adopted. Then,

$\left(a\right)$ * If either $\Lambda\left(X^{\prime}\gamma^{\ast,ipt}\right)=p\left(X\right)\leavevmode\nobreak\ a.s$ or $X^{\prime}\beta_{0,\Delta}^{\ast,wls,p}=m_{0,\Delta}^{p}\left(X\right)\leavevmode\nobreak\ a.s.$ , then, as $n\rightarrow\infty$ ,*

[TABLE]

and

[TABLE]

where $V_{imp}^{p}=\mathbb{E}\left[\eta_{imp}^{dr,p}\left(W;\gamma^{\ast,ipt},\beta_{0,\Delta}^{\ast,wls,p},\tau_{imp}^{dr,p}\right)^{2}\right]$ .

$\left(b\right)$ *If both $\Lambda\left(X^{\prime}\gamma^{\ast,ipt}\right)=p\left(X\right)\leavevmode\nobreak\ a.s$ and $X^{\prime}\beta_{0,\Delta}^{\ast,wls,p}=m_{0,\Delta}^{p}\left(X\right)\leavevmode\nobreak\ a.s.$ , then $\eta_{imp}^{dr,p}\left(W;\gamma^{\ast,ipt},\beta_{0,\Delta}^{\ast,wls,p},\tau_{imp}^{dr,p}\right)=\eta^{e,p}\left(Y_{1},Y_{0},D,X\right)\leavevmode\nobreak\ a.s.$ and $V_{imp}^{p}$ is equal to the semiparametrically efficiency bound (2.12). *

Part $\left(a\right)$ of Theorem 2 generalizes the cross-section results of Vermeulen and Vansteelandt (2015) to the DID framework. It states that the proposed DR DID estimators for the ATT, $\widehat{\tau}_{imp}^{dr,p}$ , is doubly robust, $\sqrt{n}$ -consistent and asymptotically normal. It also states that the exact form of $V_{imp}^{p}$ does not depend on which working models are correctly specified, implying that $\widehat{\tau}_{imp}^{dr,p}$ is doubly robust not only in terms of consistency but also terms of inference. An important consequence of this DR-for-inference property is that it allows one to treat the summands of $\widehat{\tau}_{imp}^{dr,p}$ as if they were independent and identically distributed, and, therefore, estimate $V_{imp}^{p}$ by

[TABLE]

Part $\left(b\right)$ of Theorem 2 indicates that $\widehat{\tau}_{imp}^{dr,p}$ is semiparametrically efficient when the working model for the propensity score, and the working models for the outcome regression for the comparison units are correctly specified.

Remark 2.

From the discussion above, it may be natural to directly use the moment conditions (3.6) to form (generic) nonlinear generalized method of moment (GMM) estimators for $\gamma$ and $\beta$ . However, it is important to emphasize that to justify the use of such estimation procedure, one must at least establish the local identification of the pseudo-true parameters, which, in turn, requires the matrix of derivatives of (3.6) having full column rank. Importantly, such a condition may not hold for some working models. This is particularly the case when one adopts the working models (3.5) and both specifications are correctly specified. Thus, care must be taken when one attempts to use alternative, more general estimation techniques to generalize the DR inference results discussed above.

Remark 3.

As discussed in Appendix A of Graham et al. (2012), it is possible to use alternative specifications for the propensity score, e.g., a probit working model. However, when one deviates from the logit specification, the optimization algorithm involved to estimate the nuisance parameters $\gamma$ tends to be more computationally demanding, as it involves numerical integration. As discussed above, $\widehat{\gamma}^{ipt}$ clearly avoids such complications.

3.2 Improved DR DID estimators when repeated cross-section data are

available

In this section, we turn our attention to our proposed improved DR DID estimators for the ATT when only repeated cross-section data are available. Similar to the panel data case, we consider the case where a researcher is comfortable with the following specifications,

[TABLE]

We consider two improved DR DID estimators based on (2.8) and (2.9), namely

[TABLE]

and

[TABLE]

where

[TABLE]

Here, note that $\widehat{\tau}_{1,imp}^{dr,rc}$ does not rely on OR models for the treated group while $\widehat{\tau}_{2,imp}^{dr,rc}$ does. In addition, when one compares $\widehat{\tau}_{1,imp}^{dr,rc}$ and $\widehat{\tau}_{2,imp}^{dr,rc}$ with $\widehat{\tau}_{imp}^{dr,p}$ , it is evident that the latter relies on a single OR model since we observe $Y_{1}$ and $Y_{0}$ for all units; when only repeated cross-section data are available, one needs to model the OR in each time period (and each treatment group). Another interesting feature worth mentioning is that we estimate the OR parameters for the treated group via ordinary least squares, whereas we estimate the OR parameters for the control group with weighted least squares. This follows from the fact that estimating the pseudo-true parameters $\beta_{1,t}^{\ast,rc}$ , $t=0,1$ , does not lead to any estimation effect, and therefore one can choose her favorite estimation method. Given this observation and the linear specification in (3.8), we find it natural to estimate $\beta_{1,t}^{\ast,rc}$ , $t=0,1$ , via OLS as this is the most widespread estimation procedure adopted by practitioners.

Let

[TABLE]

and for $\beta_{imp}^{\ast,rc}=\left(\beta_{0,1}^{\ast,wls,rc},\beta_{0,0}^{\ast,wls,rc},\beta_{1,1}^{\ast,ols,rc},\beta_{1,0}^{\ast,ols,rc}\right),$ define

[TABLE]

where $\eta_{1}^{rc,1}$ , $\eta_{0}^{rc,1}$ , $\eta_{1}^{rc,2}$ , and $\eta_{0}^{rc,2}$ are defined as in (B.1)-(B.5) in the Appendix B.

Next theorem states that $\widehat{\tau}_{1,imp}^{dr,rc}$ and $\widehat{\tau}_{2,imp}^{dr,rc}$ are not only doubly robust consistent but also doubly robust for inference. Furthermore, it states that $\widehat{\tau}_{2,imp}^{dr,rc}$ is locally semiparametrically efficient, whereas $\widehat{\tau}_{1,imp}^{dr,rc}$ is not.

Theorem 3.

Let $n=n_{1}+n_{0}$ , where $n_{1}$ and $n_{0}$ are the sample sizes of the post-treatment and pre-treatment periods, respectively. Suppose Assumptions 1-3 and Assumptions A.1-A.2 stated in Appendix A hold, that $n_{1}/n\overset{p}{\rightarrow}\lambda\in\left(0,1\right)$ as $n_{0},n_{1}\rightarrow\infty$ , and that the working nuisance models (3.8) are adopted. Then,

$\left(a\right)$ * If either $\Lambda\left(X^{\prime}\gamma^{\ast,ipt}\right)=p\left(X\right)\leavevmode\nobreak\ a.s$ or $X^{\prime}\beta_{0,1}^{\ast,wls,rc}-X^{\prime}\beta_{0,0}^{\ast,wls,rc}=m_{0,\Delta}^{rc}\left(X\right)\leavevmode\nobreak\ a.s.$ , then, for $j=1,2$ , as $n\rightarrow\infty$ ,*

[TABLE]

and

[TABLE]

where $V_{j,imp}^{rc}=\mathbb{E}\left[\eta_{j,imp}^{dr,rc}\left(W;\gamma^{\ast,ipt},\beta_{imp}^{\ast,rc}\right)^{2}\right]$ .

$\left(b\right)$ * Suppose that $\Lambda\left(X^{\prime}\gamma^{\ast,ipt}\right)=p\left(X\right)\leavevmode\nobreak\ a.s$ and, for all $\left(d,t\right)\in\left\{0,1\right\}^{2}$ , $X^{\prime}\beta_{d,t}^{\ast,wls,rc}=m_{d,t}^{rc}\left(X\right)\leavevmode\nobreak\ a.s.$ . Then, $\eta_{2,imp}^{dr,rc}\left(W;\gamma^{\ast,ipt},\beta_{imp}^{\ast,rc}\right)=\eta^{e,rc}\left(Y,D,T,X\right)$ $a.s.$ , and $V_{2,imp}^{rc}$ is equal to the semiparametrically efficiency bound (2.14). On the other hand, $V_{1,imp}^{rc}\$ does not attain the semiparametric efficiency bound. *

In other words, Theorem 3 states that both $\widehat{\tau}_{1,imp}^{dr,rc}$ and $\widehat{\tau}_{2,imp}^{dr,rc}$ are doubly robust for the ATT, $\sqrt{n}$ -consistent and asymptotically normal. Similar to the panel data case, the exact form of the $V_{j,imp}^{rc}$ , $j=1,2$ , does not depend on which working models are correctly specified, implying that both $\widehat{\tau}_{1,imp}^{dr,rc}$ and $\widehat{\tau}_{2,imp}^{dr,rc}$ are also doubly robust in terms of inference.

Part $\left(b\right)$ of Theorem 3 indicates that $\widehat{\tau}_{2,imp}^{dr,rc}$ is semiparametrically efficient when the working model for the propensity score, and all working models for the outcome regressions, for both treated and comparison units, are correctly specified. When compared to Theorem 2 $\left(b\right)$ , it is evident that such a requirement is much stronger than when panel data are available. Part $\left(b\right)$ of Theorem 3 also indicates that, in general, $\widehat{\tau}_{1,imp}^{dr,rc}$ is not locally semiparametrically efficient. As so, we argue that, in practice, one should favor $\widehat{\tau}_{2,imp}^{dr,rc}$ with respect to $\widehat{\tau}_{1,imp}^{dr,rc}$ , as both estimators are doubly robust in terms of consistency and inference, but the former is locally semiparametrically efficiency whereas the latter is not.

We conclude this section by providing a precise characterization of the efficiency loss associated with using $\widehat{\tau}_{1,imp}^{dr,rc}$ instead of $\widehat{\tau}_{2,imp}^{dr,rc}$ when all working models are correctly specified. Here, our main goal is to illustrate that by using an estimator that attempts to mimic the panel data setup and that does not explicitly exploit the stationarity condition in Assumption 1(b), one may incur in substantial efficiency loss. As so, we argue that, in practice, one should favor estimators based on the DR moment (2.9)—such as $\widehat{\tau}_{2,imp}^{dr,rc}$ —with respect to estimators based on the DR moment (2.8)—such as $\widehat{\tau}_{1,imp}^{dr,rc}$ .

Corollary 2.

Suppose the assumptions in Theorem 3 hold. Furthermore, assume that $\Lambda\left(X^{\prime}\gamma^{\ast,ipt}\right)=p\left(X\right)\leavevmode\nobreak\ a.s$ and, for all $\left(d,t\right)\in\left\{0,1\right\}^{2}$ , $X^{\prime}\beta_{d,t}^{\ast,wls,rc}=m_{d,t}^{rc}\left(X\right)\leavevmode\nobreak\ a.s.$ . Then,

[TABLE]

Remark 4.

We stress that the result in Corollary 2 does not depend on the fact that one is using the specifications in (3.8). As we show in its proof, such a result remains true provided that the (generic) first-step estimators for the nuisance functions are correctly specified. Thus, Corollary 2 quantifies the loss of efficiency associated with using estimators based on $\tau_{1}^{dr,rc}$ as defined in (2.8)—which includes the estimator proposed by Zimmert (2019)— instead of using estimators based on $\tau_{2}^{dr,rc}$ as defined in (2.9). Given that this loss of efficiency is usually strictly positive, estimators based on $\tau_{1}^{dr,rc}$ are not, in general, semiparametrically efficient. As we show in the next section via Monte Carlo simulations, this loss of efficiency can be large.

4 Monte Carlo simulation study

In this section, we conduct a series of Monte Carlo experiments in order to study the finite sample properties of our proposed DR DID estimators. When panel data are available, we compare our proposed DR DID estimators $\widehat{\tau}^{dr,p}$ and $\widehat{\tau}_{imp}^{dr,p}$ given in (3.1) and (3.7), respectively, to the OR DID estimator (2.2), the Horvitz and Thompson (1952) type IPW estimator (2.4), and the TWFE regression model (2.5). Given that the weights of the IPW estimator (2.4) are not normalized to sum up to one, $\widehat{\tau}^{ipw,p}$ can be unstable particularly when propensity score estimates are relatively close to one. To assess the role played by the weights, we also consider the Hájek (1971) type IPW estimator for the ATT

[TABLE]

where the weights $\widehat{w}_{1}^{p}\left(D\right)$ and $\widehat{w}_{0}^{p}\left(D,X;\widehat{\gamma}\right)$ are given by (3.2) and are normalized to sum up to one.

When only repeated cross-section data are available, we compare our proposed DR DID estimators $\widehat{\tau}_{1}^{dr,rc}$ and $\widehat{\tau}_{2}^{dr,rc}$ given in (3.3) and (3.4), and their further improved versions $\widehat{\tau}_{1,imp}^{dr,rc}$ and $\widehat{\tau}_{2,imp}^{dr,rc}$ given in (3.9) and (3.10), to the OR DID estimator (2.2), the plug-in IPW estimator based on (2.3), and the TWFE regression model (2.5). As in the case of panel data, we also consider the Hájek (1971) type IPW estimator for the ATT

[TABLE]

where the weights are the same as those in $\widehat{\tau}_{1}^{dr,rc}$ .

In all simulation exercises, we consider a logistic propensity score working model and a linear regression working model for the outcome evolution. All observed covariates enter the working models linearly. With the exception of $\widehat{\tau}_{imp}^{dr,p}$ , $\widehat{\tau}_{j,imp}^{dr,rc}$ , $j=1,2$ , where we use the estimation methods proposed in Section 3.1 and in Section 3.2, the OR models are estimated using ordinary least squares, and the propensity score working model is estimated using maximum likelihood estimation. When panel data are available, we consider OR models for $\Delta Y\,$ instead of OR models for $Y_{0}$ and $Y_{1}$ separately.

We consider sample size $n$ equal to $1000$ . For each design, we conduct $10,000$ Monte Carlo simulations. We compare the various DID estimators for the ATT in terms of average bias, median bias, root mean square error (RMSE), empirical 95% coverage probability, the average length of a 95% confidence interval, and the average of their plug-in estimator for the asymptotic variance. The confidence intervals are based on the normal approximation, with the asymptotic variances being estimated by their sample analogues. We also compute the semiparametric efficiency bound under each design to allow one to assess the potential loss of efficiency/accuracy associated with using inefficient DID estimators for the ATT.

4.1 Simulation 1: panel data are available

We first discuss the case where panel data are available. For a generic $W=\left(W_{1},W_{2},W_{3},W_{4}\right)^{\prime},$ let

[TABLE]

Let $\mathbf{X}=\left(X_{1},X_{2},\right.$ $\left.X_{3},X_{4}\right)^{\prime}$ be distributed as $N\left(0,I_{4}\right)$ , and $I_{4}$ be the $4\times 4$ identity matrix. For $j=1,2,3,4$ , let $Z_{j}=\left.\left(\tilde{Z}-\mathbb{E}\left[\tilde{Z}\right]\right)\right/\sqrt{Var\left(\tilde{Z}\right)}$ , where $\tilde{Z}_{1}=\exp\left(0.5X_{1}\right)$ , $\tilde{Z}_{2}=10+X_{2}/\left(1+\exp\left(X_{1}\right)\right)$ , $\tilde{Z}_{3}=\left(0.6+\left.X_{1}X_{3}\right/25\right)^{3}$ and $\tilde{Z}_{4}=\left(20+X_{2}+X_{4}\right)^{2}$ .

Building on Kang and Schafer (2007), we consider the following data generating processes (DGPs):

[TABLE]

where $\varepsilon_{0}$ , $\varepsilon_{1}\left(d\right)$ , $d=0,1$ are independent standard normal random variables, $U$ is an independent standard uniform random variable, and for a generic $W$ , $v\left(W,D\right)$ is an independent normal random variable with mean $D\cdot f_{reg}\left(W\right)$ and variance one. The available data are $\left\{Y_{0,i},Y_{1,i},D_{i},Z_{i}\right\}_{i=1}^{n}$ , where $Y_{0}=Y_{0}\left(0\right)$ , and $Y_{1}=DY_{1}\left(1\right)+\left(1-D\right)Y_{1}\left(0\right)$ . In the aforementioned DGPs, the true ATT is zero, and $v$ plays the role of time-invariant unobserved heterogeneity.

Given that we focus on the empirically relevant setting where the observed covariates $Z$ enter all working models linearly, it is clear that in $DPG1$ , both propensity score (PS) and OR working models are correctly specified. In $DGP2$ , only the OR working model is correctly specified, whereas in $DGP3$ only the PS working model is correctly specified. In $DGP4$ , all working models are misspecified. The simulation results are presented in Table 1.

First, note that the TWFE estimator $\widehat{\tau}^{fe}$ is severely biased and its confidence interval for the ATT has almost zero coverage in all analyzed DGPs. These results should not be unexpected, because, as discussed in Remark 1, $\widehat{\tau}^{fe}$ implicitly rules out covariate-specific trends, and when these are relevant, like in the considered DGPs, the estimand associated with $\widehat{\tau}^{fe}$ is not the ATT. As so, policy evaluations based on $\widehat{\tau}^{fe}$ can be misleading.

The results in Table 1 also suggest that, when both the OR and PS working models are correctly specified, all semiparametric estimators for the ATT show little to no Monte Carlo bias, but $\widehat{\tau}^{reg}$ , $\widehat{\tau}^{dr,p}$ and $\widehat{\tau}_{imp}^{dr,p}$ dominate the IPW DID estimators $\widehat{\tau}^{ipw,p}$ and $\widehat{\tau}_{std}^{ipw,p}$ on the basis of bias, root mean square error, asymptotic variance, and length of the confidence interval. Indeed, both IPW DID estimator seem to be substantially less efficient than $\widehat{\tau}^{reg}$ , $\widehat{\tau}^{dr,p}$ and $\widehat{\tau}_{imp}^{dr,p}$ . The performance of these last three estimators are very close, though $\widehat{\tau}^{reg}$ tends to be more efficient than the other two DR DID estimators. Given that $\widehat{\tau}^{reg}$ exploits additional assumptions when compared to $\widehat{\tau}^{dr,p}$ and $\widehat{\tau}_{imp}^{dr,p}$ , such a result is not unexpected. Also note that the Hájek (1971) type IPW estimator $\widehat{\tau}_{std}^{ipw,rc}$ is more stable than the Horvitz and Thompson (1952) type IPW estimator $\widehat{\tau}^{ipw,rc}$ : the RMSE (the asymptotic variance) of $\widehat{\tau}^{ipw,rc}$ are more than two (four) times bigger than that of $\widehat{\tau}_{std}^{ipw,rc}$ . Such a finding highlights the practical importance of using weights that are normalized to sum up to one.

When only the OR working model is correctly specified, our proposed DR DID estimators $\widehat{\tau}^{dr,p}$ and $\widehat{\tau}_{imp}^{dr,p}$ are competitive with the OR DID estimator $\widehat{\tau}^{reg}$ , while the IPW DID estimators are biased, as one should expect. On the other hand, when only the PS working model is correctly specified, the IPW and DR estimators show little to no bias, while $\widehat{\tau}^{reg}$ displays non-negligible bias. Here, it is worth emphasizing that $\widehat{\tau}^{dr,p}$ and $\widehat{\tau}_{imp}^{dr,p}$ drastically outperform $\widehat{\tau}^{ipw,p}$ and $\widehat{\tau}_{std}^{ipw,p}$ , with $\widehat{\tau}_{imp}^{dr,p}$ also showing substantial improvements with respect to both $\widehat{\tau}^{dr,p}$ and $\widehat{\tau}_{std}^{ipw,p}$ . When one compares the two IPW estimators, the role played by the normalized weights is again clear, as $\widehat{\tau}_{std}^{ipw,p}$ is again much more “stable” than $\widehat{\tau}^{ipw,p}$ .

When both OR and PS working models are misspecified, not unexpectedly all estimators have non-negligible biases and inference procedures are, in general, misleading. In this scenario, our DR DID estimators have smaller biases and RMSE than the OR and the normalized IPW DID estimators, with $\widehat{\tau}_{imp}^{dr,p}$ strictly dominating $\widehat{\tau}^{dr,p}.$ However, the Horvitz and Thompson (1952) IPW DID estimator $\widehat{\tau}^{ipw,p}$ seems to perform best in this DGP.

In terms of efficiency, the results in Table 1 show that the estimated asymptotic variance of $\widehat{\tau}^{dr,p}$ and $\widehat{\tau}_{imp}^{dr,p}$ are very close to the semiparametric efficiency bound when both the PS and OR regression are correctly specified, which is in agreement with our locally efficiency results in Theorems 2 and A.1 (in the Appendix). When the PS is misspecified but the OR is not, the estimated asymptotic variances of $\widehat{\tau}^{dr,p}$ and $\widehat{\tau}_{imp}^{dr,p}$ are still close to the semiparametric efficiency bound in this particular DGP, though we emphasize that this is not predicted by our results and can be a feature of this particular DGP. Finally, we note that when the OR is misspecified but the PS is not, the estimated asymptotic variances of our proposed DR DID estimators $\widehat{\tau}^{dr,p}$ and $\widehat{\tau}_{imp}^{dr,p}$ are far from the semiparametric efficiency bound, with $\widehat{\tau}_{imp}^{dr,p}$ outperforming $\widehat{\tau}^{dr,p}$ in terms of efficiency in this particular DGP.

4.2 Simulation 2: repeated cross-section data are available

We now analyze the performance of the DID estimators for the ATT when one only observes repeated cross-section data. To do so, we consider the same DGPs as in the panel data framework, but instead of observing data on $\left(Y_{0},Y_{1},D,Z\right),$ one observes data on $\left(Y_{0},D,Z\right)$ if $T=0$ , or on $\left(Y_{1},D,Z\right)$ if $T=1$ , where $T=1\left\{U_{T}\leq\lambda\right\}$ , and $U_{T}$ is a standard uniform random variable, and $\lambda\in\left(0,1\right)$ a fixed constant.

Table 2 presents the simulation results with $\lambda=0.5$ and with $n\equiv n_{1}+n_{0}=1,000$ .888Simulation results with $\lambda=0.25$ and $\lambda=0.75$ reached analogous conclusions to those discussed below and are available upon request. Overall, the simulation exercise reveals that the efficiency bound, RMSE, asymptotic variance, and confidence interval length of the considered DID estimators are much larger when only repeated cross-section data are available than when panel data are available. In light of Corollary 1, such a result should be expected, though the magnitude of such loss of efficiency can be striking. In addition, the results in Table 2 reveal that: $\left(i\right)$ the TWFE estimator $\widehat{\tau}^{fe}$ is severely biased for the ATT in all DGPs, just like in the panel data case; $\left(ii\right)$ the IPW estimator with standardized weights $\widehat{\tau}_{std}^{ipw,rc}$ is much more stable and efficient than $\widehat{\tau}^{ipw,rc}$ in all DGPs, and, as one should expect, when the PS working model is misspecified, these IPW estimators display non-negligible biases; $\left(iii\right)$ as one should expect, the OR DID estimator displays non-negligible bias when the OR working models are misspecified; $\left(iv\right)$ all four DR DID estimators display little to no bias when one of the working models is correctly specified, but the locally efficient DR DID estimators $\widehat{\tau}_{2}^{dr,rc}$ and $\widehat{\tau}_{2,imp}^{dr,rc}$ present important efficiency gains when compared to all other DID estimators, including $\widehat{\tau}_{1}^{dr,rc}$ and $\widehat{\tau}_{1,imp}^{dr,rc}$ . These gains in efficiency are more pronounced when the OR models are correctly specified. The simulation results also show that $\left(v\right)$ when one compares the performance of the further improved DR DID estimators $\widehat{\tau}_{1,imp}^{dr,rc}$ and $\widehat{\tau}_{2,imp}^{dr,rc}$ with the “traditional” DR DID estimators $\widehat{\tau}_{1}^{dr,rc}$ and $\widehat{\tau}_{2}^{dr,rc}$ , it is clear that appropriately choosing the estimation methods for the nuisance parameters can have practical consequences, especially when the outcome regression working models are misspecified.

In terms of efficiency, the results in Table 2 highlight that, when all working models are correctly specified, the estimated asymptotic variances of $\widehat{\tau}_{2}^{dr,rc}$ and $\widehat{\tau}_{2,imp}^{dr,rc}$ are indeed close to the semiparametric efficiency bound, but the asymptotic variances $\widehat{\tau}_{1}^{dr,rc}$ and $\widehat{\tau}_{1,imp}^{dr,rc}$ are substantially higher than the semiparametric efficiency bound; these findings are in agreement with our locally efficiency results in Theorems 3 and A.2 (in the Appendix). Similarly to the panel data case, we find that, in this specific DGP, the estimated asymptotic variances of $\widehat{\tau}_{2}^{dr,rc}$ and $\widehat{\tau}_{2,imp}^{dr,rc}$ are still close to the semiparametric efficiency bound when the outcome regressions are correctly specified but the PS is not, but not when the PS is correctly specified but the outcome regressions are not.

5 Empirical illustration: the effect of job training on earnings

In a very influential study, LaLonde (1986) analyzes whether different treatment effect estimators based on observational data are able to replicate the experimental findings of the NSW job training program on post treatment earnings. His negative results led to an increased awareness of the potential pitfalls of observational data and helped spur the use of randomized controlled trials among economists. In addition, alternative policy evaluation tools arose to overcome “LaLonde’s critique” of observational estimators. Two prominent examples are the propensity score matching (PSM), see e.g. Dehejia and Wahba (1999, 2002) (henceforth DW) and the difference-in-differences matching, see e.g. Heckman et al. (1997) and Smith and Todd (2005) (henceforth ST). For instance, DW show that PSM can replicate the experimental benchmark of the NSW for a particular subsample of the original data. ST, on the other hand, cast doubt on the “generalizability” of DW PSM results to a larger population and argue that the conclusions may be sensitive to the propensity score specification. ST also argue that for the NSW data, difference-in-differences matching estimators may be more suitable than cross-section PSM, as they can account for time-invariant unobserved confounding factors.

Motivated by ST findings, in what follows, we focus on DID estimators and evaluate whether our proposed DR DID estimators can better reduce the selection bias when compared to other DID estimation procedures. We analyze three different experimental samples — the original LaLonde experimental sample, the DW sample, and the “early random assignment” (early RA) subsample of the DW sample considered by ST — and consider data from the Current Population Survey (CPS) to form a non-experimental comparison group. The pre-treatment covariates in the data include age, years of education, real earnings in 1974, and dummy variables for high school dropout, married, black, and Hispanic. The outcome of interest is real earnings in 1978. We also observe real earnings in 1975, which we use as the pre-treatment outcome $Y_{0}$ . The experimental benchmark for the ATT is equal to $\$ 886 $(s.e.$ $488 $),$ $1794 $(s.e.$ $671 $), and$ $2748 $(s.e.$ $1005$) for the LaLonde, DW, and early RA sample, respectively. For additional description and summary statistics for each sample, see Smith and Todd (2005).

Following ST, we focus on estimating the average “evaluation bias” of different DID estimators. This is only made possible given the availability of experimental data. First, randomization ensures that both “treatment” groups are comparable in terms of self selection. Second, given that randomized-out individuals did not receive training via NSW, the impact of NSW is known to be zero in this group. Thus, applying different DID estimators to data from randomized-out individuals (our pseudo treated group in this exercise) and nonexperimental CPS comparison observations (our comparison group in this exercise) should produce an estimated ATT equal to zero, if these DID estimators are consistent. Deviations from zero are what we call evaluation bias.999An alternative way to estimate “evaluation bias” is to compare the ATT using the experimental data with ATT using data from randomized-in and nonexperimental comparison units. This is the approach taken by LaLonde (1986) and Dehejia and Wahba (1999, 2002). A disadvantage of this approach compared to the one we and Smith and Todd (2005) use is that experimental ATT estimates are also random and may differ from the “true” ATT. Thus, the computation of “true” evaluation biases is much more challenging if not impossible. In any case, results treating the experimental ATT as true effects lead to similar conclusions and are available upon request.

Like in the Monte Carlo simulation exercises, we compare our proposed DR DID estimators $\widehat{\tau}^{dr,p}$ and $\widehat{\tau}_{imp}^{dr,p}$ with the TWFE estimator $\widehat{\tau}^{fe}$ based on (2.5), the OR DID estimator $\widehat{\tau}^{reg}$ as defined in (2.2), and the Horvitz and Thompson (1952) type IPW DID estimator proposed by Abadie (2005), $\widehat{\tau}^{ipw,p}$ , as defined in (2.4). We also consider the Hájek (1971) type IPW estimator $\widehat{\tau}_{std}^{ipw,p}$ as defined in (4.1). We assume that the outcome models are linear in parameters and that the propensity score follows a logistic specification. The unknown parameters are estimated using ordinary least squares (OLS) and maximum likelihood, respectively, except in $\widehat{\tau}_{imp}^{dr,p}$ , where we use the estimation methods described in Section 3.1.

In order to assess the sensitivity of the findings with respect to the model specifications, we consider three different specifications for how covariates enter into each model: $\left(i\right)$ a linear specification where all covariates enter the models linearly; $\left(ii\right)$ a specification in the spirit of DW, which adds to the linear specification a dummy for zero earnings in 1974, age squared, age cubed divided by $1000$ , years of schooling squared, and an interaction term between years of schooling and real earnings in 1974; and $\left(iii\right)$ an “augmented DW” specification, which adds to the “DW” specification the interactions between married and real earnings in 1974, and between married and zero earnings in 1974 — these two interaction terms were used in Firpo (2007).

Table LABEL:tab:st-bias summarizes the results. Standard errors are reported in parentheses and the estimated evaluation biases relative to the experimental ATT benchmark are reported in brackets. As argued by ST, these “relative biases” are useful for comparing DID estimators within each sample, but as the experimental benchmark estimates for the ATT vary substantially among the three experimental samples, they should not be used for comparing DID estimators across samples.

Table LABEL:tab:st-bias highlights some interesting patterns. First, estimators based on two-way fixed effect regression models tend to be very stable across specifications, but usually display large positive and statistically significant evaluation biases. Second, DID estimators based on the regression approach tend to lead to the most precise estimates. However, for the LaLonde sample, point estimates are severely biased downward, leading to statistically significant evaluation biases. Abadie’s IPW estimators $\widehat{\tau}^{ipw,p}$ for the ATT tend to have the largest standard errors across all considered estimators, but their evaluation biases are relatively small. Like in our Monte Carlo simulation results, considering normalized weights as in $\widehat{\tau}_{std}^{ipw,p}$ can improve the stability of the IPW estimators $\widehat{\tau}^{ipw,p}$ . Finally, note that our proposed DR DID estimators share the favorable bias properties of Abadie’s IPW estimator, but at the same time, have smaller standard errors than IPW estimators. When we compare $\widehat{\tau}^{dr,p}$ with $\widehat{\tau}_{imp}^{dr,p}$ , we note that the further improved DR DID estimator $\widehat{\tau}_{imp}^{dr,p}$ tends to have smaller standard errors, particularly when one adopts the “DW” or the “augmented DW” specifications. Taken together, the results using the NSW job training data suggest that our proposed DR DID estimators are an attractive alternative to existing DID procedures.

6 Concluding remarks

In this article, we proposed doubly robust estimators for the ATT in difference-in-differences settings where the parallel trends assumption holds only after conditioning on a vector of pre-treatment covariates. Our proposed estimators remain consistent for the ATT when either (but not necessarily both) a propensity score model or outcome regression models are correctly specified, and achieve the semiparametric efficiency bound when the working models for the nuisance functions are correctly specified. We derived the large sample properties of the proposed estimators in situations where either panel data or repeated cross-section data are available, and showed that by paying particular attention to the estimation methods used to estimate the nuisance parameters, one can form DID estimators for the ATT that are not only DR consistent and locally semiparametric efficient, but also DR for inference. We illustrated the attractiveness of our proposed causal inference tools via a simulation exercise and with an empirical application.

Our results can be extended to other situations of practical interest. A leading case is when researchers are interested in understanding treatment effect heterogeneity with respect to continuous covariates $X_{1}$ , where $X_{1}$ is a (strict) subset of available covariates $X$ . Here, the parameter of interest is the conditional average treatment effect on the treated CATT $\left(X_{1}\right)\equiv\mathbb{E}\left[Y\left(1\right)-Y\left(0\right)|X_{1},D=1\right]$ and because of its infinite dimensional nature, the estimation and inference tools proposed in this paper are not directly applicable. However, by combining the DR DID formulation proposed in this paper with the methodology put forward by Chen and Christensen (2018), one can propose uniformly valid inference procedures not only for the CATT but also for possibly nonlinear functionals of the CATT such as (higher order) partial derivatives, conditional average (higher order) partial derivatives, and partial derivatives of its $log$ .

Another interesting extension is when researchers want to adopt data-adaptive, “machine learning” first-step estimators instead of the parametric models discussed in this paper. Here, the main challenge is to derive the influence function of the DR DID estimator for the ATT, as “machine learning” estimators are, in general, in a non-Donsker classes of functions. We envision that one can bypass such technical complications by combining the results derived in this paper with those in Chernozhukov et al. (2017), Belloni et al. (2017), and Tan (2019), for example; see e.g. Zimmert (2019) for some recent results in this direction. We leave the detailed analysis of these extensions to future work.

A Appendix A: Asymptotic Properties of the DR DID estimators based on

generic first-step estimators

Let $g\left(x\right)$ be a generic notation for $\pi\left(x\right)$ , $\mu_{d,t}^{p}\left(x\right)$ and $\mu_{d,t}^{rc}\left(x\right)$ , $d,t=0,1$ . Analogously and with some abuse of notation, let $g\left(x;\theta\right)$ be a generic notation for $\pi\left(x;\gamma\right)$ , $\mu_{d,t}^{p}\left(x,\beta_{d,t}^{p}\right)$ and $\mu_{d,t}^{rc}\left(x,\beta_{d,t}^{rc}\right)$ , $d,t=0,1$ . Let $W=\left(Y_{0},Y_{1},D,X\right)$ in the panel data case and $W=\left(Y,T,D,X\right)$ in the repeated cross-section data case. Denote the support of $X$ by $\mathcal{X}$ and for a generic $Z$ , let $\left\|Z\right\|=\sqrt{trace\left(Z^{\prime}Z\right)}$ denote the Euclidean norm of $Z$ .

Let

[TABLE]

where $\kappa^{p}=\left(\gamma^{\prime},\beta_{0,0}^{p\prime},\beta_{0,1}^{p\prime}\right)^{\prime}$ , $\kappa^{rc\leavevmode\nobreak\ 1}=\left(\gamma^{\prime},\beta_{0,0}^{rc\prime},\beta_{0,1}^{rc\prime}\right)^{\prime}$ and $\kappa^{rc\leavevmode\nobreak\ 2}=\left(\gamma^{\prime},\beta_{0,0}^{rc\prime},\beta_{0,1}^{rc\prime},\beta_{1,1}^{rc\prime},\beta_{1,0}^{rc\prime}\right)^{\prime}$ . In obvious notation, the vector of pseudo-true parameter101010Note that we allow for possible misspecification when we define pseudo-true parameters. is given by $\kappa^{\ast,p}$ , $\kappa_{0}^{\ast,rc\leavevmode\nobreak\ 1}$ , and $\kappa^{\ast,rc\leavevmode\nobreak\ 2}$ . Let $\dot{h}^{p}\left(W;\kappa^{p}\right)=\partial\left.h^{p}\left(W;\kappa^{p}\right)\right/\partial\kappa^{p}$ and define $\dot{h}^{rc\leavevmode\nobreak\ j}\left(W;\kappa^{rc\leavevmode\nobreak\ j}\right)$ , $j=0,1$ , analogously.

Assumption A.1.

$\left(i\right)$ * $g\left(x\right)=g\left(x;\theta\right)$ is a parametric model, where $\theta\in\Theta\subset\mathbb{R}^{k}$ , $\Theta$ being compact; $\left(ii\right)$ $g\left(X;\theta\right)$ is $a.s.$ continuous at each $\theta$ $\in\Theta$ ; $\left(iii\right)$ there exists a unique pseudo-true parameter $\theta^{\ast}\in int\left(\Theta\right)$ ; $\left(iv\right)$ $g\left(X;\theta\right)$ is $a.s.$ twice continuously differentiable in a neighborhood of $\theta^{\ast}$ , $\Theta^{\ast}\subset\Theta$ ; $\left(v\right)$ the estimator $\widehat{\theta}$ is strongly consistent for the $\theta^{\ast}$ and satisfies the following linear expansion:*

[TABLE]

*where $l_{g}\left(\cdot;\theta\right)$ is such that $\mathbb{E}\left[l_{g}\left(W;\theta^{\ast}\right)\right]=0$ , $\mathbb{E}\left[l_{g}\left(W;\theta^{\ast}\right)l_{g}\left(W;\theta^{\ast}\right)^{\prime}\right]$ exists and is positive definite and $\lim_{\delta\rightarrow 0}\mathbb{E}\left[\sup_{\theta\in\Theta^{\ast}:\left\|\theta-\theta^{\ast}\right\|\leq\delta}\left\|l_{g}\left(W;\theta\right)-l_{g}\left(W;\theta^{\ast}\right)\right\|^{2}\right]=0$ . In addition, $\left(vi\right)$ for some $\varepsilon>0,$ $0<\pi\left(X;\gamma\right)\leq 1-\varepsilon$ $a.s.$ , for all $\gamma\in int\left(\Theta^{ps}\right)$ , where $\Theta^{ps}$ denotes the parameter space of $\gamma.$ *

Assumption A.2.

$\left(i\right)$ * When panel data are available, assume that $\mathbb{E}\left[\left\|h^{p}\left(W;\kappa^{\ast,p}\right)\right\|^{2}\right]<\infty$ and $\mathbb{E}\left[\sup_{\kappa\in\Gamma^{\ast,p}}\left|\dot{h}^{p}\left(W;\kappa\right)\right|\right]<\infty,$ where $\Gamma^{\ast,p}$ is a small neighborhood of $\kappa^{\ast,p}$ . $\left(ii\right)$ When cross-section data are available, assume that, for $j=1,2$ , $\mathbb{E}\left[\left\|h^{rc,j}\left(W;\kappa^{\ast,rc,j}\right)\right\|^{2}\right]<\infty$ and $\mathbb{E}\left[\sup_{\kappa\in\Gamma^{\ast,rc\leavevmode\nobreak\ j}}\left|\dot{h}^{rc,j}\left(W;\kappa\right)\right|\right]<\infty,$ where $\Gamma^{\ast,rc\leavevmode\nobreak\ \leavevmode\nobreak\ j}$ is a small neighborhood of $\kappa^{\ast,rc\leavevmode\nobreak\ j}$ . *

Assumptions A.1-A.2 are standard in the literature, see e.g. Abadie (2005), Wooldridge (2007), Bonhomme and Sauder (2011), Graham et al. (2012) and Callaway and Sant’Anna (2018). Assumption A.1 requires that the first-step estimators are based on smooth parametric models and that the estimated parameters admit $\sqrt{n}$ -asymptotically linear representations, whereas Assumption A.2 imposes some weak integrability conditions. Under mild moment conditions, these requirements are fulfilled when one adopts linear/nonlinear outcome regressions or logit/probit models, for example, and estimates the unknown parameters by (nonlinear) least squares, quasi-maximum likelihood, or other alternative estimation methods, see e.g. Chapter 5 in van der Vaart (1998), Wooldridge (2007), Graham et al. (2012) and Sant’Anna et al. (2018).

Next, we derive the asymptotic properties of $\widehat{\tau}^{dr,p}$ , $\widehat{\tau}_{1}^{dr,rc}$ and $\widehat{\tau}_{2}^{dr,rc}$ using generic first-step estimators that satisfy Assumptions A.1 and A.2.

A.1 Panel data case

In this section, we discuss the asymptotic properties of $\widehat{\tau}^{dr,p}$ . Define $\dot{\pi}\left(x;\gamma\right)\equiv\left.\partial\pi\left(x;\gamma\right)\right/\partial\gamma$ and, for $t=0,1$ , define $\dot{\mu}_{0,t}^{p}\left(x;\beta_{0,t}^{p}\right)$ analogously. In what follows, we drop the dependence of the functionals on $W$ to ease the notational burden. For example, we write $w_{1}^{p}=w_{1}^{p}\left(D\right)$ , $w_{0}^{p}\left(\gamma\right)=w_{0}^{p}\left(D,X;\gamma\right)$ , and so on and so forth.

For generic $\gamma$ and $\beta_{0}=\left(\beta_{0,1}^{{}^{\prime}},\beta_{0,0}^{{}^{\prime}}\right)^{\prime}$ , let

[TABLE]

where

[TABLE]

and

[TABLE]

with $l_{reg}\left(\beta_{0}\right)=\left(l_{reg,0,1}\left(\beta_{0,1}\right)^{\prime},l_{reg,0,0}\left(\beta_{0,0}\right)^{\prime}\right)^{\prime}$ , where $l_{reg,d,t}\left(\cdot\right)$ is the asymptotic linear representation of the estimators for the outcome regression as described in Assumption A.1 $\left(iv\right)$ , $l_{ps}\left(\cdot\right)$ is defined analogously, $\dot{\mu}_{0,\Delta}^{p}\left(\beta_{0}\right)=\left(\dot{\mu}_{0,1}^{p}\left(\beta_{0,1}\right)^{\prime},\leavevmode\nobreak\ -\dot{\mu}_{0,0}^{p}\left(\beta_{0,0}\right)\right)^{\prime}$ and

[TABLE]

For $d,t=0,1$ , let $\Theta_{d,t}^{reg}\$ be the parameter space for the regression coefficient $\beta_{d,t}$ , and $\Theta^{ps}$ be the parameter space for the propensity score coefficient $\gamma$ . Consider the following claims:

[TABLE]

Now we are ready to state the large sample properties of $\widehat{\tau}^{dr,p}$ .

Theorem A.1.

Suppose Assumptions 1-3 and Assumptions A.1-A.2 stated in Appendix A hold.

$\left(a\right)$ * Provided that either (A.3) or (A.4) is true, as $n\rightarrow\infty$ ,*

[TABLE]

Furthermore,

[TABLE]

where $V^{p}=\mathbb{E}[\eta^{p}\left(W;\gamma^{\ast},\beta_{0}^{\ast,p}\right)^{2}]$ .

$\left(b\right)$ * When both (A.3) and (A.4) are true, $\eta^{p}\left(W;\gamma^{\ast},\beta_{0}^{\ast,p}\right)=\eta^{e,p}\left(Y_{1},Y_{0},D,X\right)$ $a.s.$ and $V^{p}$ is equal to the semiparametrically efficiency bound (2.12). *

Theorem A.1 indicates that, provided that either the propensity score model or the model for the evolution of the outcome for the comparison group is correctly specified, $\widehat{\tau}^{dr,p}$ is consistent for the ATT, implying that our proposed estimator is indeed doubly robust. In addition, Theorem A.1 indicates that our proposed estimator admits an asymptotically linear representation and as a consequence, it is $\sqrt{n}$ -consistent and asymptotically normal. When the models for the nuisance functions are correctly specified, our proposed DR DID estimator is semiparametrically efficient.

Theorem A.1 also suggests that one can use the analogy principle to estimate $V^{p}$ and conduct asymptotically valid inference.111111It is easy to show that the plug-in estimator of $V^{p}$ is consistent, see e.g. Lemma 4.3 in Newey and McFadden (1994) and Theorem 4.4 in Abadie (2005). We omit the detailed derivation of this result for the sake of brevity. However, it is worth mentioning the fact that the exact form of $V^{p}$ depends on which nuisance models are correctly specified, implying that our (generic) estimator $\widehat{\tau}^{dr,p}$ is doubly robust in terms of consistency but, in general, not doubly robust for inference. Given that in practice it is hard to know a priori which nuisance models are correctly specified, one should include all “correction” terms in $\eta_{est}^{p}$ when estimating $V^{p}$ . Failing to do so may lead to asymptotically invalid inference procedures.

A.2 Repeated cross-section data case

In this section, we turn our attention to our proposed DR DID estimators for the ATT when only repeated cross-section data are available. For generic $\gamma$ and $\beta=\left(\beta_{1}^{{}^{\prime}},\beta_{0}^{{}^{\prime}}\right)^{\prime}$ , where, for $d=0,1,$ $\beta_{d}=\left(\beta_{d,1}^{{}^{\prime}},\beta_{d,0}^{{}^{\prime}}\right)^{\prime}$ , let

[TABLE]

such that, for $j=1,2$ ,

[TABLE]

and the precise definitions of all these $\eta^{rc}$ functions are deferred to Appendix B to avoid excess notational complexity. An aspect of the difference between $\eta_{1}^{rc}$ and $\eta_{2}^{rc}$ that is worth mentioning but is perhaps buried in the notation is that $\eta_{1}^{rc}$ depends on $\beta$ only through $\beta_{0}$ , whereas $\eta_{2}^{rc}$ depends on both $\beta_{1}$ and $\beta_{0}$ . This is simply a consequence from the fact that $\widehat{\tau}_{1}^{dr,rc}$ does not rely on outcome regressions for the treated units, but $\widehat{\tau}_{2}^{dr,rc}$ does.

Consider the following claims:

[TABLE]

Theorem A.2.

Let $n=n_{1}+n_{0}$ , where $n_{1}$ and $n_{0}$ are the sample sizes of the post-treatment and pre-treatment periods, respectively. Suppose Assumptions 1-3 and Assumptions A.1-A.2 stated in Appendix A hold, and that $n_{1}/n\overset{p}{\rightarrow}\lambda\in\left(0,1\right)$ as $n_{0},n_{1}\rightarrow\infty$ .

$\left(a\right)$ * Provided that either (A.3) or (A.6) is true, as $n\rightarrow\infty$ , for $j=1,2$ ,*

[TABLE]

Furthermore,

[TABLE]

where $V_{j}^{rc}=\mathbb{E}[\eta_{j}^{rc}\left(W;\gamma^{\ast},\beta^{\ast,rc}\right)^{2}]$ .

$\left(b\right)$ * Suppose that both (A.3) and (A.7) are true. Then, $\eta_{2}^{rc}\left(W;\gamma^{\ast},\beta^{\ast,rc}\right)=\eta^{e,rc}\left(Y,D,T,X\right)$ $a.s.$ , and $V_{2}^{rc}$ is equal to the semiparametrically efficiency bound (2.14). On the other hand, $V_{1}^{rc}\$ does not attain the semiparametric efficiency bound when (A.3) and (A.7) are true. *

In other words, Theorem A.2 states that both proposed estimators for the ATT, $\widehat{\tau}_{1}^{dr,rc}$ and $\widehat{\tau}_{2}^{dr,rc}$ , are doubly robust, $\sqrt{n}$ -consistent and asymptotically normal. Similar to the panel data case, the exact form of the $V_{j}^{rc}$ , $j=1,2$ , depends on which working models are correctly specified, implying that the generic estimators $\widehat{\tau}_{1}^{dr,rc}$ and $\widehat{\tau}_{2}^{dr,rc}$ are doubly robust in terms of consistency but in terms of inference.

Part $\left(b\right)$ of Theorem A.2 indicates that $\widehat{\tau}_{2}^{dr,rc}$ is semiparametrically efficient when the working model for the propensity score, and all working models for the outcome regressions, for both treated and comparison units, are correctly specified. When compared to Theorem A.1 $\left(b\right)$ , it is evident that such a requirement is stronger than when panel data are available.

B Appendix B: Influence function of the DR DID estimators with

repeated cross-section

As it is evident from Theorem A.2, the influence functions of $\widehat{\tau}_{1}^{dr,rc}$ and $\widehat{\tau}_{2}^{dr,rc}$ play a major role in study of the large sample properties of our proposed DR DID estimators. In this section, we state the precise definition of $\eta_{j}^{rc}\left(W;\gamma,\beta\right)$ , $j=1,2$ , introduced in (A.5).

We first focus on $\widehat{\tau}_{1}^{dr,rc}$ . For generic $\gamma$ and $\beta=\left(\beta_{1}^{{}^{\prime}},\beta_{0}^{{}^{\prime}}\right)^{\prime}$ , where, for $d=0,1,$ $\beta_{d}=\left(\beta_{d,1}^{{}^{\prime}},\beta_{d,0}^{{}^{\prime}}\right)^{\prime}$ , let

[TABLE]

where

[TABLE]

and, for $t=0,1$ ,

[TABLE]

and the influence functions associated with the estimation effects of the nuisance parameters are

[TABLE]

and

[TABLE]

where, for $t=0,1$ ,

[TABLE]

and $w_{1,t}^{rc}\equiv w_{1,t}^{rc}\left(D,T\right)$ , $w_{0,t}^{rc}(\gamma)\equiv w_{0,t}^{rc}(D,T,X;\gamma)$ .

The influence function of $\widehat{\tau}_{2}^{dr,rc}$ is given by

[TABLE]

where

[TABLE]

and, for $d=0,1$ , $\mu_{d,\Delta}^{rc}\left(X;\beta_{d,1},\beta_{d,0}\right)\equiv\mu_{d,1}^{rc}\left(X;\beta_{d,1}\right)-\mu_{d,0}^{rc}\left(X;\beta_{d,0}\right)$ , and

[TABLE]

Note that estimating the OR coefficients associated with the treated group does not lead to any estimation effect.

Bibliography51

The reference list from the paper itself. Each links out to its DOI / PubMed record.

1(1)
2Abadie (2005) Abadie, A. (2005), “Semiparametric difference-in-difference estimators,” Review of Economic Studies , 72, 1–19.
3Ai and Chen (2003) Ai, C., and Chen, X. (2003), “Efficient estimation of models with conditional moment restrictions containin unknown functions,” Econometrica , 71(6), 1795–1843.
4Ai and Chen (2007) Ai, C., and Chen, X. (2007), “Estimation of possibly misspecified semiparametric conditional moment restriction models with different conditioning variables,” Journal of Econometrics , 141(1), 5–43.
5Ai and Chen (2012) Ai, C., and Chen, X. (2012), “The semiparametric efficiency bound for models of sequential moment restrictions containing unknown functions,” Journal of Econometrics , 170(2), 442–457.
6Angrist and Pischke (2009) Angrist, J. D., and Pischke, J.-S. (2009), Mostly Harmless Econometrics: An Empiricist ’ s Companion , Princeton, NJ: Princeton University Press.
7Bang and Robins (2005) Bang, H., and Robins, J. M. (2005), “Doubly robust estimation in missing data and causal inference models,” Biometrics , 61(4), 962–972.
8Belloni et al. (2017) Belloni, A., Chernozhukov, V., Fernández-Val, I., and Hansen, C. (2017), “Program Evaluation and Causal Inference With High-Dimensional Data,” Econometrica , 85(1), 233–298.

TL;DR

Contribution

Findings

Abstract

Peer Reviews

Videos

Doubly Robust Difference-in-Differences Estimators††thanks: First complete version: November 29, 2018. We thank the editor, Serena Ng, the

Abstract

1 Introduction

2 Difference-in-differences

2.1 Background

Assumption 1**.**

Assumption 2**.**

Assumption 3**.**

Remark 1**.**

2.2 Doubly robust difference-in-differences estimands

Theorem 1**.**

2.3 Semiparametric efficiency bound

Proposition 1**.**

Corollary 1**.**

3 Estimation and inference

3.1 Improved DR DID estimators when panel data are available

Theorem 2**.**

Remark 2**.**

Remark 3**.**

3.2 Improved DR DID estimators when repeated cross-section data are

Theorem 3**.**

Corollary 2**.**

Remark 4**.**

4 Monte Carlo simulation study

4.1 Simulation 1: panel data are available

4.2 Simulation 2: repeated cross-section data are available

5 Empirical illustration: the effect of job training on earnings

6 Concluding remarks

A Appendix A: Asymptotic Properties of the DR DID estimators based on

Assumption A.1**.**

Assumption A.2**.**

A.1 Panel data case

Theorem A.1**.**

A.2 Repeated cross-section data case

Theorem A.2**.**

B Appendix B: Influence function of the DR DID estimators with

Assumption 1.

Assumption 2.

Assumption 3.

Remark 1.

Theorem 1.

Proposition 1.

Corollary 1.

Theorem 2.

Remark 2.

Remark 3.

Theorem 3.

Corollary 2.

Remark 4.

Assumption A.1.

Assumption A.2.

Theorem A.1.

Theorem A.2.