Characterization of parameters with a mixed bias property

TL;DR

This paper studies a class of parameters with a 'mixed bias property' that allows for robust estimation in non-parametric models, including causal inference parameters, by characterizing their influence functions and deriving functional equations.

Contribution

It introduces and characterizes the class of parameters with the mixed bias property, expanding on existing classes and providing tools for robust estimation.

Findings

The mixed bias property class includes many parameters of interest in causal inference.

Derived influence functions and functional equations for parameters with this property.

Proposed loss functions enable penalized estimation of nuisance functions.

Abstract

In this article we study a class of parameters with the so-called `mixed bias property'. For parameters with this property, the bias of the semiparametric efficient one step estimator is equal to the mean of the product of the estimation errors of two nuisance functions. In non-parametric models, parameters with the mixed bias property admit so-called rate doubly robust estimators, i.e. estimators that are consistent and asymptotically normal when one succeeds in estimating both nuisance functions at sufficiently fast rates, with the possibility of trading off slower rates of convergence for the estimator of one of the nuisance functions with faster rates for the estimator of the other nuisance. We show that the class of parameters with the mixed bias property strictly includes two recently studied classes of parameters which, in turn, include many parameters of interest in causal inference. We characterize the form of parameters with the mixed bias property and of their influence functions. Furthermore, we derive two functional moment equations, each being solved at one of the two nuisance functions, as well as, two functional loss functions, each being minimized at one of the two nuisance functions. These loss functions can be used to derive loss based penalized estimators of the nuisance functions.

Equations

$$\sqrt{n}\left\{\hat{\chi }-\chi \left(\eta \right)\right\}=\sqrt{n}\left\{\chi \left(\hat{\eta }\right)-\chi \left(\eta \right)+E_{\eta }\left({\chi }_{\hat{\eta }}^1\right)\right\}+{\mathbb{G}}_n\left({\chi }_{\hat{\eta }}^1-{\chi }_{\eta }^1\right)+{\mathbb{G}}_n\left({\chi }_{\eta }^1\right).$$

$$\chi \left(\hat{\eta }\right)-\chi \left(\eta \right)-E_{\eta }\left({\chi }_{\hat{\eta }}^1\right)$$

$$\chi \left({\eta }^{\prime }\right)-\chi \left(\eta \right)+E_{\eta }\left({\chi }_{{\eta }^{\prime }}^1\right)=E_{\eta }\left[S_{ab}\left\{a^{\prime }\left(Z\right)-a\left(Z\right)\right\}\left\{b^{\prime }\left(Z\right)-b\left(Z\right)\right\}\right]$$

$$\chi \left(\eta \right)=E_{\eta }\left\{m_1\left(O,a\right)\right\}+E_{\eta }\left(S_0\right)=E_{\eta }\left\{m_2\left(O,b\right)\right\}+E_{\eta }\left(S_0\right)$$

$${\chi }_{\eta }^1=S_{ab}a\left(Z\right)b\left(Z\right)+m_1(O,a)+m_2(O,b)+S_0-\chi \left(\eta \right)$$

$$h\in L_2\left(P_{\eta ,Z}\right)\to E_{\eta }\left\{m_1\left(O,h\right)\right\}\text{ and }h\in L_2\left(P_{\eta ,Z}\right)\to E_{\eta }\left\{m_2\left(O,h\right)\right\}$$

$$a=\arg \underset{h\in L_2\left(P_{\eta ,Z}\right)}{\min }{E}_{\eta }\left\{S_{ab}\frac{h^2}{2}+m_2\left(O,h\right)\right\}\text{ and }b=\arg \underset{h\in L_2\left(P_{\eta ,Z}\right)}{\min }{E}_{\eta }\left\{S_{ab}\frac{h^2}{2}+m_1\left(O,h\right)\right\}$$

$$S_{ab}\left\{a\left(Z,\eta \right)-a\left(Z,{\eta }^{\prime }\right)\right\}\left\{b\left(Z,\eta \right)-b\left(Z,{\eta }^{\prime }\right)\right\}=S_{ab}^{\ast }\left\{a^{\ast }\left(Z^{\ast },\eta \right)-a^{\ast }\left(Z^{\ast },{\eta }^{\prime }\right)\right\}\left\{b^{\ast }\left(Z^{\ast },\eta \right)-b^{\ast }\left(Z^{\ast },{\eta }^{\prime }\right)\right\}.$$

$$p\left(y|D=0,Z\right)=p\left(y|D=1,Z\right)\exp \left(\delta y\right)/E\left\{\exp \left(\delta Y\right)|D=1,Z\right\}$$

$${\frac{d}{dt}\chi \left({\eta }_t\right)|}_{t=0}={\frac{d}{dt}\chi \left(a_t,b,\tau \right)|}_{t=0}+{\frac{d}{dt}\chi \left(a,b_t,\tau \right)|}_{t=0}+{\frac{d}{dt}\chi \left(a,b,{\tau }_t\right)|}_{t=0}$$

$$\chi \left(a_t,b,\tau \right)=E_{\left(a_t,b,\tau \right)}\left\{\chi \left({\eta }^{\prime }\right)+{\chi }_{{\eta }^{\prime }}^1\right\}$$

$$\chi \left(a,b_t,\tau \right)=E_{\left(a,b_t,\tau \right)}\left\{\chi \left({\eta }^{\prime }\right)+{\chi }_{{\eta }^{\prime }}^1\right\}$$

$$\chi \left(a,b,{\tau }_t\right)=E_{\left(a,b,{\tau }_t\right)}\left\{\chi \left({\eta }^{\prime }\right)+{\chi }_{{\eta }^{\prime }}^1\right\}$$

$${\frac{d}{dt}\chi \left({\eta }_t\right)|}_{t=0}$$

$${\chi }_{\eta }^1$$

$$S_0^{\ast }\equiv {\left(\chi +{\chi }^1\right)}_{\left(a^{\ast },b^{\ast }\right)}-S_{ab}{a}^{\ast }{b}^{\ast }$$

$$m_1^{\ast }\left(O,a\right)\equiv \left\{{\left(\chi +{\chi }^1\right)}_{\left(a,b^{\ast }\right)}-S_{ab}a{b}^{\ast }\right\}-S_0^{\ast }$$

$$m_2^{\ast }\left(O,b\right)\equiv \left\{{\left(\chi +{\chi }^1\right)}_{\left(a^{\ast },b\right)}-S_{ab}{a}^{\ast }b\right\}-S_0^{\ast }$$

$$E_{\eta }\left[m_1^{\ast }\left(O,h\right)\right]$$

$$E_{\eta }\left\{m_2^{\ast }\left(O,h\right)\right\}=-E_{\eta }\left(S_{ab}ha\right)+E_{\eta }\left(S_{ab}a{b}^{\ast }\right)$$

$${\chi }_{\eta }^1=S_{ab}ab+m_1^{\ast }\left(O,a\right)+m_2^{\ast }\left(O,b\right)+S_0^{\ast }-\chi \left(\eta \right)$$

$$E_{\eta }\left\{m_1^{\ast }\left(O,a\right)+m_2^{\ast }\left(O,b\right)+S_0^{\ast }\right\}$$

$$E_{\eta }\left\{S_{ab}{a}_{t}b+m_1^{\ast }\left(O,a_t\right)\right\}=E_{\eta }\left(S_{ab}{a}^{\ast }b\right)$$

$$E_{\eta }\left\{S_{ab}a{b}_t+m_2^{\ast }\left(O,b_t\right)\right\}=E_{\eta }\left(S_{ab}a{b}^{\ast }\right)$$

$${\frac{d}{dt}{E}_{\eta }\left\{S_{ab}{a}_t{b}_t+m_1^{\ast }\left(O,a_t\right)+m_2^{\ast }\left(O,b_t\right)+S_0^{\ast }\right\}|}_{t=0}$$

$${\chi }_{\eta }^1=S_{ab}ab+m_1^{\ast \ast }\left(O,a\right)+m_2^{\ast \ast }\left(O,b\right)+S_0^{\ast \ast }-\chi \left(\eta \right).$$

$$m_1^{\ast \ast }\left(O,a\right)+m_2^{\ast \ast }\left(O,b\right)+S_0^{\ast \ast }=m_1^{\ast }\left(O,a\right)+m_2^{\ast }\left(O,b\right)+S_0^{\ast }.$$

$$m_1^{\ast \ast }\left(O,a\right)-m_1^{\ast }\left(O,a\right)=m_2^{\ast }\left(O,b\right)-m_2^{\ast \ast }\left(O,b\right)+S_0^{\ast }-S_0^{\ast \ast }.$$

$$E_{\eta }\left(Q_1^{†}\right)$$

$$E_{\eta }\left\{m_1^{†}\left(O,h\right)\right\}$$

Full text

1 Introduction

Suppose that we are given a sample $\mathcal{D}_{n}$ of $n$ i.i.d. copies of a random vector $O$ with law $P$ which is known to belong to $\mathcal{M}=\left\{P_{\eta}:\eta\in\mathbf{\eta}\right\}$ where $\mathbf{\eta}$ is a large, non-Euclidean, parameter space. Our goal is to estimate the value taken by a scalar parameter $\chi\left(\eta\right)$ at $P$. Suppose $O$ includes a vector $Z$ with sample space $\mathcal{Z}$ $\subset R^{d}.$ We are interested in parameters $\chi\left(\eta\right)$ which cannot be estimated without estimating some unknown function of the covariates $Z,$ such as a conditional mean given $Z$ or a density of $Z.$

Given an initial estimator $\widehat{\eta}$, the plug-in estimator $\chi\left(\widehat{\eta}\right)$ is a natural choice for estimating $\chi\left(\eta\right).$ However, except for special estimators $\widehat{\eta}$ targeted to specific parameters $\chi\left(\eta\right)$, $\chi\left(\widehat{\eta}\right)$ is not $\sqrt{n}$- consistent. A strategy for reducing the bias of $\chi\left(\widehat{\eta}\right)$ is to subtract from it an estimate $-\mathbb{P}_{n}\chi_{\widehat{\eta}}^{1}$ of its first order bias where for each $\eta,$ $\chi_{\eta}^{1}\equiv\chi_{\eta}^{1}\left(O\right)$ is an adequately chosen random variable and $\mathbb{P}_{n}h$ is the empirical mean operator $n^{-1}\sum_{i=1}^{n}h\left(O_{i}\right)$. This strategy yields the one step estimator $\widehat{\chi}\equiv\chi\left(\widehat{\eta}\right)+\mathbb{P}_{n}\chi_{\widehat{\eta}}^{1}.$ A good choice for $\chi_{\eta}^{1}$ is a so called influence function of $\chi\left(\eta\right)$. See for example Newey et al., (1998), Newey et al., (2004) and Robins et al., (2017). Heuristically, this choice is guided by the following analysis. Write

[TABLE]

where $E_{\eta}\left(\chi_{\widehat{\eta}}^{1}\right)\equiv\int\chi_{\widehat{\eta}}^{1}\left(o\right)dP_{\eta}\left(o\right)$ and $\mathbb{G}_{n}\left(\chi_{\widehat{\eta}}^{1}\right)\equiv$ $\sqrt{n}\mathbb{P}_{n}\left\{\chi_{\widehat{\eta}}^{1}-E_{\eta}\left(\chi_{\widehat{\eta}}^{1}\right)\right\}.$

The term $\mathbb{G}_{n}\left(\chi_{\eta}^{1}\right)$ is $\sqrt{n}$ times the sample average of mean zero random variables, so it converges to a normal distribution. On the other hand, if model $\mathcal{M}$ is not too big, then for estimators $\widehat{\eta}$ converging to $\eta$ one may expect $\mathbb{G}_{n}\left(\chi_{\widehat{\eta}}^{1}-\chi_{\eta}^{1}\right)$ to be $o_{p}\left(1\right)$. One can make this term $o_{p}\left(1\right)$ even without restrictions on model size by employing the following strategy known as cross-fitting (Schick,, 1986; Van der Vaart,, 2000; Ayyagari,, 2010; Zheng and van der Laan,, 2011) . First split sample $\mathcal{D}_{n}$ into two samples, next compute $\widehat{\eta}$ from one subsample and the one step estimator from the other subsample. Next, compute a second one step estimator by repeating the procedure but switching the roles of the two subsamples. Finally, compute the estimator $\widetilde{\chi}$ of $\chi\left(\eta\right)$ as the average of both one step estimators. Convergence of $\sqrt{n}\left\{\widetilde{\chi}-\chi\left(\eta\right)\right\}$ to a mean zero normal distribution thus depends essentially solely on

[TABLE]

being $o_{p}\left(n^{-1/2}\right).$ This last requirement suggests that we choose $\chi_{\eta}^{1}$ to be an influence function of $\chi\left(\eta\right)$. This is because for such choice $E_{\eta}\left(\chi_{\widehat{\eta}}^{1}\right)$ acts like minus the derivative of $\chi\left(\eta\right)$ in the direction $\widehat{\eta}-\eta$. Consequently $\left(\ref{eq:expansion}\right)$ acts like the residual from a first order Taylor’s expansion of $\chi\left(\eta\right),$ and hence is of order $O\left(\left\|\widehat{\eta}-\eta\right\|^{2}\right).$ Thus, for estimators $\widehat{\eta}$ such that $n^{1/4}\left\|\widehat{\eta}-\eta\right\|=o_{p}\left(1\right)$ for some norm$\left\|\cdot\right\|,$ $\left(\ref{eq:expansion}\right)$ should be of order $o_{p}\left(n^{-1/2}\right).$ See Chapter 25 in Van der Vaart, (2000) for the definition of influence functions. Parameters that admit influence functions are called regular parameters. Such parameters have a unique influence function if the model $\mathcal{M}$ is non-parametric. By non-parametric we mean that the closed linear span of the scores for all parametric submodels at $P$ of model $\mathcal{M}$ is equal to $L_{2}\left(P\right).$ Throughout we will assume that $\mathcal{M}$ is non-parametric, that $\chi\left(\eta\right)$ is regular and that $\chi_{\eta}^{1}$ is the unique influence function of $\chi\left(\eta\right)$.

Many parameters $\chi\left(\eta\right)$ of interest in Causal Inference and Econometrics have influence functions which satisfy the following property.

Definition 1** (Mixed bias property)**

For each $\eta$ there exist functions $a\left(Z\right)\equiv a\left(Z;\eta\right)$ and $b\left(Z\right)\equiv b\left(Z;\eta\right)$ such that for any $\eta^{\prime}:$

[TABLE]

where $a^{\prime}\left(Z\right)\equiv a\left(Z;\eta^{\prime}\right),b^{\prime}\left(Z\right)\equiv b\left(Z;\eta^{\prime}\right)$ and $S_{ab}\equiv s_{ab}\left(O\right)$ and $o\rightarrow s_{ab}\left(o\right)$ is a known function, i.e. it that does not depend on $\eta.\vskip 6.0pt plus 2.0pt minus 2.0pt$

As we will see in the next section, the mixed bias property implies that $\chi\left(\eta\right)+\chi_{\eta}^{1}$ depends on $\eta$ only through $a$ and $b$ and, consequently, the one step estimator depends on $\widehat{\eta}$ only through estimators $\widehat{a}$ and $\widehat{b}$. The property implies that for estimators $\widehat{a}$ and $\widehat{b}$ satisfying $\int\left\{\widehat{a}\left(z\right)-a\left(z\right)\right\}^{2}dP_{\eta}\left(z\right)=O_{p}\left(\gamma_{a,n}\right)$ and $\int\left\{\widehat{b}\left(z\right)-b\left(z\right)\right\}^{2}dP_{\eta}\left(z\right)=O_{p}\left(\gamma_{b,n}\right)$, equation $\left(\ref{eq:expansion}\right)$ is of order $O_{P}\left(\gamma_{a,n}\gamma_{b,n}\right).$ This in turn implies that, when cross-fitting is employed, $\widetilde{\chi}$ has the so-called rate double robustness property in that $\sqrt{n}\left\{\widetilde{\chi}-\chi\left(\eta\right)\right\}$ converges to a mean zero Normal distribution if $\gamma_{a,n}=o\left(1\right),$ $\gamma_{b,n}=o\left(1\right)$ and $\gamma_{a,n}\gamma_{b,n}=o\left(n^{-1/2}\right).$ Because the rates of convergence $\gamma_{a,n}$ and $\gamma_{b,n}$ of estimators $\widehat{a}$ and $\widehat{b}$ depend on the complexities of $a$ and $b,$ then for parameters $\chi\left(\eta\right)$ satisfying the mixed bias property, $\sqrt{n}\left\{\widetilde{\chi}-\chi\left(\eta\right)\right\}$ is asymptotically normal even if one of the functions $a$ or $b$ is very complex so long as the other is simple enough.

Recent articles have identified two distinct classes of parameters $\chi\left(\eta\right)$ with the mixed bias property and gave many examples of parameter of interest in causal inference and econometrics, including the examples in § 4 below. The first class, described in Robins et al., (2008) is comprised of parameters with influence function of the form $\chi_{\eta}^{1}=S_{ab}a\left(Z\right)b\left(Z\right)+S_{a}a\left(Z\right)+S_{b}b\left(Z\right)+S_{0}-\chi\left(\eta\right)$ where $S_{a}$ and $S_{b}$ are statistics. The second class, described in Chernozhukov et al., 2018b (see also Hirshberg and Wager, (2017) and Chernozhukov et al., 2018a ) is comprised of parameters of the form $\chi\left(\eta\right)=E_{\eta}\left\{d\left(O,a\right)\right\}$ where $a\left(Z\right)\equiv E_{\eta}\left(Y|Z\right)$ and $d\left(O,a\right)$ is such that the map $h\in L_{2}\left(P_{\eta,Z}\right)\rightarrow E_{\eta}\left\{d\left(O,h\right)\right\}$ is continuous and affine linear.

In this paper we will characterize the form of the influence function, under a non-parametric model $\mathcal{M}$, of any parameter satisfying the mixed bias property. We will show that the class of parameters satisfying the mixed bias property strictly includes the union of the Robins et al., (2008) and Chernozhukov et al., 2018b classes. Furthermore, we will show that neither the class Robins et al., (2008) nor that of Chernozhukov et al., 2018b is contained in the other. We will also show that, under mild regularity conditions, parameters that satisfy the mixed bias property are necessarily of the form

[TABLE]

for some statistic $S_{0}$, and some $m_{1}$ and $m_{2}$ such that the maps $h\in\mathcal{A}\rightarrow m_{1}\left(O,h\right)$ and $h\in\mathcal{B}\rightarrow m_{2}\left(O,h\right)$ are linear, where $\mathcal{A}\equiv\left\{a\left(Z;\eta\right):\eta\in\mathbf{\eta}\right\}$ and $\mathcal{B}\equiv\left\{b\left(Z;\eta\right):\eta\in\mathbf{\eta}\right\}.$ In addition, we will prove a number of results about the structure of $a$ and/or $b$ in special cases. In particular, we will show that, under mild regularity conditions, when $a$ does not depend on the marginal distribution of $Z,$ then, up to regularity conditions, a necessary and sufficient condition for $\chi\left(\eta\right)$ to have the mixed bias property is that $\chi\left(\eta\right)=E_{\eta}\left\{m_{1}\left(O,a\right)\right\}+E_{\eta}\left(S_{0}\right)$ for a statistic $S_{0},$ a linear map $h\in\mathcal{A}\rightarrow m_{1}\left(O,h\right)$ and $a\left(Z\right)$ a ratio of two conditional means given $Z.$ We will also show that for parameters $\chi\left(\eta\right)$ that satisfy the mixed bias property the influence function naturally yields two loss functions whose expectations are minimized at $a$ and $b$ respectively. These loss functions can then be used to construct loss-based machine-learning estimators of $a$ and $b$ such as support vector machine estimators (Christmann and Steinwart,, 2008).

Our work is related to Robins and Rotnitzky, (2001) and Chernozhukov et al., (2016). These papers discuss sufficient conditions for the existence of, so called, doubly robust estimating functions. A key distinction of our work is that, unlike these papers, we do not assume that the parameter solves a population moment equation, rather we deduce this fact from the primitive condition of the mixed bias property.

2 Characterization of the influence functions with the mixed bias

property

Our first result establishes that for parameters $\chi\left(\eta\right)$ that satisfy the mixed bias property, $\chi\left(\eta\right)+\chi_{\eta}^{1}$ depends on $\eta$ only through $a$ and $b.$

Proposition 1

If $\chi\left(\eta\right)$ satisfies the mixed bias property and the regularity Condition 1, then $\chi\left(\eta\right)+\chi_{\eta}^{1}$   depends on $\eta$ only through $a$ and $b.$

The Supplementary Material contains proofs of all the claims made in this article, the regularity Conditions 1 and 2 invoked by them, and further examples of parameters with the mixed bias property. The next Theorem characterizes the influence functions of parameters with the mixed bias property.

Theorem 1

If $\chi\left(\eta\right)$ satisfies the mixed bias property and the regularity Condition 1 holds, then there exist a statistic $S_{0}$ and maps $h\in\mathcal{A}\rightarrow$ $m_{1}\left(O,h\right)$ and $h\in\mathcal{B}\rightarrow m_{2}\left(O,h\right)$, independent of $\eta$, such that the maps $h\in\mathcal{A}\rightarrow E_{\eta}\left\{m_{1}\left(O,h\right)\right\}$ and $h\in\mathcal{B}\rightarrow E_{\eta}\left\{m_{2}\left(O,h\right)\right\}$ are linear and such that $\left(\ref{eq:parameter}\right)$ and

[TABLE]

hold. Furthermore, for all $h\in\mathcal{A},$ $E_{\eta}\left\{S_{ab}hb+m_{1}\left(O,h\right)\right\}=0$ and for all $h\in\mathcal{B},$ $E_{\eta}\left\{S_{ab}ha+m_{2}\left(O,h\right)\right\}=0.$ In addition, for any $h_{1},h_{2}\in\mathcal{A}$ and constants $\alpha_{1},\alpha_{2}$ such that $h\equiv\alpha_{1}h_{1}+\alpha_{2}h_{2}\in\mathcal{A}$ and such that $m_{1}\left(O,h\right),m_{1}\left(O,h_{1}\right)\,$ and $m_{1}\left(O,h_{2}\right)$ are in $L_{2}\left(P_{\eta}\right)$, it holds that $m_{1}\left(O,h\right)=\alpha_{1}m_{1}\left(O,h_{1}\right)+\alpha_{2}m_{1}\left(O,h_{2}\right)$ a.s.$\left(P_{\eta}\right).$ In particular, if for all $h\in\mathcal{A}$, $m_{1}\left(O,h\right)\in L_{2}\left(P_{\eta}\right)$ then the map $h\in\mathcal{A}\rightarrow$ $m_{1}\left(O,h\right)$ is linear a.s.$\left(P_{\eta}\right)$. Likewise, if for all $h\in\mathcal{B}$ it holds that $m_{2}\left(O,h\right)\in L_{2}\left(P_{\eta}\right)$ then the map $h\in\mathcal{B}\rightarrow$ $m_{2}\left(O,h\right)$ is linear a.s.$\left(P_{\eta}\right)$.

Part (i) of the next result establishes that under a slightly stronger requirement on $m_{1}$ and $m_{2}$ and some regularity conditions, the reverse of Theorem 1 also holds. The theorem also establishes several additional results that we will comment after its statement.

Theorem 2

Suppose that for each $\eta$ there exist functions $a\left(Z\right)\equiv a\left(Z;\eta\right)$ and $b\left(Z\right)\equiv b\left(Z;\eta\right)$ such that the regularity Condition 1 holds and such that the influence function of $\chi\left(\eta\right)$ is of the form $\left(\ref{eq:IF}\right)$ for $m_{1}$ and $m_{2}$ that satisfy that for each $\eta,$ the maps

[TABLE]

*are continuous and linear with Riesz representers $\mathcal{R}_{1}\left(Z\right)$ and $\mathcal{R}_{2}\left(Z\right)$ respectively. Moreover, suppose $E_{\eta}\left\{m_{1}\left(O,a\right)\right\}$ and $E_{\eta}\left\{m_{2}\left(O,b\right)\right\}$ exist. Furthermore, suppose that for each $\eta,$ $E_{\eta}\left(S_{ab}|Z\right)a\left(Z\right)$ and $E_{\eta}\left(S_{ab}|Z\right)b\left(Z\right)$ are in $L_{2}\left(P_{\eta,Z}\right)$. Then,

(i) the identity $\left(\ref{bias1}\right)$ holds for each $\eta^{\prime}$ such that $a^{\prime}\left(Z\right)\equiv a\left(Z;\eta^{\prime}\right),b^{\prime}\left(Z\right)\equiv b\left(Z;\eta^{\prime}\right)$ satisfy that $a^{\prime}-a\in L_{2}\left(P_{\eta,Z}\right)$ and $b^{\prime}-b\in L_{2}\left(P_{\eta,Z}\right)$.

(ii) for all $h\in L_{2}\left(P_{\eta,Z}\right)$ it holds that $E_{\eta}\left\{S_{ab}ha+m_{2}\left(O,h\right)\right\}=0$ and $E_{\eta}\left\{S_{ab}hb+m_{1}\left(O,h\right)\right\}=0$.

(iii) if $E_{\eta}\left(S_{ab}|Z\right)\not=0$ a.s.$\left(P_{\eta,Z}\right),$ then $a\left(Z\right)=-\mathcal{R}_{2}\left(Z\right)/E_{\eta}\left(S_{ab}|Z\right).$ Likewise, if $E_{\eta}\left(S_{ab}|Z\right)\not=0$ a.s.$\left(P_{\eta,Z}\right),$ then $b\left(Z\right)=-\mathcal{R}_{1}\left(Z\right)/E_{\eta}\left(S_{ab}|Z\right).$

(iv)

(iv.a) if $a\in$ $L_{2}\left(P_{\eta,Z}\right)$ or if $\ \exists\varepsilon>0$ such that $\left(1+t\right)a\in\mathcal{A}$ for $0<t<\varepsilon$ or for $-\varepsilon<t<0$ then $\chi\left(\eta\right)=E_{\eta}\left\{m_{2}\left(O,b\right)\right\}+E_{\eta}\left(S_{0}\right).$

(iv.b) Likewise, if $b\in$ $L_{2}\left(P_{\eta,Z}\right)$ or if $\ \exists\varepsilon>0$ such that $\left(1+t\right)b\in\mathcal{B}$ for $0<t<\varepsilon$ or for $-\varepsilon<t<0$ then $\chi\left(\eta\right)=E_{\eta}\left\{m_{1}\left(O,a\right)\right\}+E_{\eta}\left(S_{0}\right).$

(iv.c) if the conditions of parts (iv.a) and (iv.b) hold then $\chi\left(\eta\right)=-E_{\eta}\left(S_{ab}ab\right)+E_{\eta}\left(S_{0}\right).$

(v) if $a\in$ $L_{2}\left(P_{\eta,Z}\right),$ $b\in$ $L_{2}\left(P_{\eta,Z}\right)$ and $E_{\eta}\left(S_{ab}|Z\right)>0$ a.s.$\left(P_{\eta,Z}\right),$ then*

[TABLE]

Note that part (ii) of Theorem 2 provides unbiased moment equations for $a$ and $b$ respectively without requiring that $a$ or $b$ be in $L_{2}\left(P_{\eta,Z}\right)$. Chernozhukov et al., 2018b and Smucler et al., (2019) exploit these moment equations to construct $\ell_{1}$ regularized estimators of the nuisance functions. Part (iii) of the theorem provides the formulae for $a$ and $b$ in terms of the Riesz representers of the maps. Part (iv) shows that under a strengthening on the requirements on $a$ and $b,$ the representation in $\left(\ref{eq:parameter}\right)$ holds. Note that the requirement that $\left(1+t\right)b\in\mathcal{B}$ for $0<t<\varepsilon$ is rather mild. For instance, for $b\left(Z\right)=1/P\left(D=1|Z\right),\,$as in example 1 below, the requirement is satisfied since the only restriction  the elements $b^{\prime}$ of $\mathcal{B}$ satisfy is that for each $z,$ $b^{\prime}\left(z\right)$ must be greater than or equal 1. Part (v) of the Theorem could in principle be used to derive other machine learning, loss-based estimators of these parameters, such as support vector machines (Christmann and Steinwart,, 2008).

3 Characterization of the nuisance functions

An interesting question is what can be said about the restrictions that the nuisance functions $a$ and $b$ of parameters with the mixed bias property must satisfy. In this section we explore this question in the special case in which $a$ does not depend on the marginal law of $Z$. We will show that such $a$ must be a ratio of conditional expectations given $Z.$

Proposition 2

Suppose that the parameter $\chi\left(\eta\right)$ satisfies the mixed bias property, the regularity Conditions 1 and 2 hold and $E_{\eta}\left(S_{ab}|Z\right)\not=0$ a.s. $\left(P_{\eta,Z}\right).$ If $a$ depends on $\eta$ only through the law of $O|Z,$ then there exists a statistic $q\left(O\right)$ such that $a\left(Z\right)=-$ $E_{\eta}\left\{q\left(O\right)|Z\right\}/E_{\eta}\left(S_{ab}|Z\right)$. Furthermore, the influence function of $\chi\left(\eta\right)$ satisfies $\left(\ref{eq:IF}\right)$ for some linear map $h\in\mathcal{A}\rightarrow m_{1}\left(O,h\right)$ and $m_{2}\left(O,b\right)=q\left(O\right)b.$

Proposition 3

Suppose that $a\left(Z\right)=-$ $E_{\eta}\left\{q\left(O\right)|Z\right\}/E_{\eta}\left(S_{ab}|Z\right)$ is in $L_{2}\left(P_{\eta,Z}\right).$ Suppose also that the map $h\in L_{2}\left(P_{\eta,Z}\right)\rightarrow E_{\eta}\left\{m_{1}\left(O,a\right)\right\}$ is linear and continuous with Riesz representer $\mathcal{R}_{1}\left(Z\right).$ Then, $\chi\left(\eta\right)=E_{\eta}\left\{m_{1}\left(O,a\right)\right\}+E_{\eta}\left(S_{0}\right)$ has influence function that satisfies $\left(\ref{eq:IF}\right)$ with $m_{2}\left(O,b\right)=q\left(O\right)b$ and $b\left(Z\right)=-{\mathcal{R}_{1}\left(Z\right)}/{E_{\eta}\left(S_{ab}|Z\right)}$. In addition, if $E_{\eta}\left\{q\left(O\right)|Z\right\}\in L_{2}\left(P_{\eta,Z}\right)$, then $\chi\left(\eta\right)$ has the mixed bias property.

For a given parameter $\chi\left(\eta\right)$ there can exist more than one function $a\left(Z\right)\equiv a\left(Z,\eta\right)$ independent of the law of $Z$ such that the mixed bias property holds for some $b\left(Z\right)\equiv b\left(Z,\eta\right).$ An instance is the parameter in Example 2 below, since $a\left(Z\right)$ can be either $E_{\eta}\left(Y|Z\right)$ or $E_{\eta}\left(D|Z\right).$ However, in that example, if $a=E_{\eta}\left(Y|Z\right)$ then $b=E_{\eta}\left(D|Z\right)$ and vice versa, if $a=E_{\eta}\left(D|Z\right)$ then $b=E_{\eta}\left(Y|Z\right).$ An open question is whether or not there exist two distinct triplets $\left(S_{ab},a,b\right)$ and $\left(S_{ab}^{\ast},a^{\ast},b^{\ast}\right)$ with $\left(a,b\right)\not=\left(a^{\ast},b^{\ast}\right)$ such that the parameter $\chi\left(\eta\right)\,\,$satisfies the mixed bias property for both triplets. This is important because if such distinct triplets existed, then there would exist two different pairs of nuisance functions of the same covariate $Z$ that one could choose to estimate in order to construct rate doubly robust estimators of $\chi\left(\eta\right)$.

In the preceding propositions we have assumed a given partition of the data $O$ into a given ‘covariate’ vector $Z$ and the remaining variables in $O.$ Interestingly, there exist parameters $\chi\left(\eta\right)$ that satisfy the mixed bias property for two different partitions of $O,$ one with ‘covariate’ vector $Z$ and another with a different ‘covariate’ vector $Z^{\ast}.$ Specifically, in Example 1 we show that for $\chi\left(\eta\right)$ equal to the mean of an outcome missing at random, there exist two possible partitions of $O,$ into two different ‘covariate’ vectors $Z$ and $Z^{\ast},$ and functions $a^{\ast}\left(Z^{\ast},\eta\right)$ and $b^{\ast}\left(Z^{\ast},\eta\right)$ different from $a\left(Z\right)$ and $b\left(Z\right),$ $a\left(Z,\eta\right)$ and $a^{\ast}\left(Z^{\ast},\eta\right)$ depending on $\eta$ only through the law of $O|Z$ such that for all $\eta$ and $\eta^{\prime}$

[TABLE]

Consequently, the parameter $\chi\left(\eta\right)$ satisfies the mixed bias property for the functions $a$ and $b,$ but also for the functions $a^{\ast}$ and $b^{\ast}.$ In this example, $S_{ab}$ is not a constant but $S_{ab}^{\ast}$ is a constant, so in view of part (i) of Proposition 2, $a\left(Z\right)$ is a ratio of two conditional expectations given $Z,$ whereas $a^{\ast}$ is a conditional expectation of a specific statistic $q\left(O\right)$ given $Z^{\ast}.$ This example raises the following interesting question: suppose that $\chi\left(\eta\right)$ satisfies the mixed bias property for a function $a\left(Z\right)$ that is a strict ratio of two conditional expectations given $Z,$ is it always possible to find a different covariate vector $Z^{\ast}$ such that $\chi\left(\eta\right)$ satisfies the mixed bias property for a function $a^{\ast}\left(Z^{\ast}\right)$ that is a conditional mean of a statistic given $Z^{\ast}?$ The answer is negative, as Example 3 below illustrates. This example proves that the class of parameters that satisfy the mixed bias property strictly includes the class considered in Chernozhukov et al., 2018b .

We conclude our analysis answering the question of whether a characterization exist of nuisance functions that depend on the marginal law of $Z$ and possibly also on the law of $O|Z$. The answer to this question is negative. This can be understood from Proposition 3 because when the map $h\in L_{2}\left(P_{\eta,Z}\right)\rightarrow E_{\eta}\left\{m_{1}\left(O,h\right)\right\}$ is linear and continuous, $b\left(Z\right)=-{\mathcal{R}_{1}\left(Z\right)}/{E_{\eta}\left(S_{ab}|Z\right)}$ where $\mathcal{R}_{1}\left(Z\right)$ is the Riesz representer of the map. The representer $\mathcal{R}_{1}\left(Z\right)$ can be many different functionals of the marginal law of $Z,$ depending on the map it represents. The examples in the next section illustrate this point.

4 Examples

In this section we present several examples of parameters satisfying the mixed bias property. These examples demonstrate that the class of parameters with the mixed bias property strictly includes the classes of Chernozhukov et al., 2018b and of Robins et al., (2008) and that neither of this classes is included in the other. In the Supplementary Material we provide further examples.

In the following examples the parameters are, possibly some function of, parameters that are in both the class of Chernozhukov et al., 2018b and of Robins et al., (2008)

Example 1** (Mean of an outcome that is missing at random and average

treatment effect)**

Suppose $O=\left(DY,D,Z\right)$ where $D$ is binary, $Y$ is an outcome which is observed if and only if $D=1$ and $Z$ is a vector of always observed covariates. If we make the untestable assumption that the density $p\left(y|D=0,Z\right)$ is equal to the density $p\left(y|D=1,Z\right),$ i.e. that the outcome $Y$ is missing at random then, for $P=P_{\eta}$, the mean of $Y$ is equal to $\chi\left(\eta\right)=E_{\eta}\left\{a\left(Z\right)\right\}$ where $a\left(Z\right)\equiv E_{\eta}\left(DY|Z\right)/E_{\eta}\left(D|Z\right)$. If $a\left(Z\right)\in L_{2}\left(P_{\eta,Z}\right)$ and $E_{\eta}\left(D|Z\right)>0,$ then the parameter $\chi\left(\eta\right)$ satisfies the conditions of Proposition 3 with $m_{1}\left(O,h\right)\equiv h$ and $S_{0}=0.$ The map $h\in L_{2}\left(P_{\eta,Z}\right)\rightarrow E_{\eta}\left\{m_{1}\left(O,h\right)\right\}$ is continuous with Riesz representer $\mathcal{R}_{1}\left(Z\right)=1,$ and $a\left(Z\right)=-E_{\eta}\left\{q\left(O\right)|Z\right\}/E_{\eta}\left(S_{ab}|Z\right)$ for $S_{ab}=-D$ and $q\left(O\right)=DY.$ Consequently, $\chi\left(\eta\right)$ has the mixed bias property for $a\left(Z\right)$ as defined and $b\left(Z\right)=1/E_{\eta}\left(D|Z\right).$ Since $m_{1}\left(O,a\right)=a$, Proposition 2 implies that $\chi\left(\eta\right)$ is in the class of parameters considered by Robins et al., (2008). Interestingly, as shown in Chernozhukov et al., 2018b and anticipated in the previous section, the parameter $\chi\left(\eta\right)$ is also in the class of Chernozhukov et al., 2018b , but for a different ‘covariate’ $Z^{\ast}.$ Specifically, let $Z^{\ast}\equiv$ $\left(D,Z\right)$ and $a^{\ast}\left(Z^{\ast}\right)\equiv E_{\eta}\left(DY|Z^{\ast}\right).$ Then, we can re-express $\chi\left(\eta\right)$ as $\chi\left(\eta\right)=E_{\eta}\left\{m_{1}^{\ast}\left(O,a^{\ast}\right)\right\}$ where for any $h^{\ast}\left(D,Z\right),$ $m_{1}^{\ast}\left(O,h^{\ast}\right)\equiv h^{\ast}\left(D=1,Z\right).$ The map $h^{\ast}\in L_{2}\left(P_{\eta,\left(D,Z\right)}\right)\rightarrow E_{\eta}\left\{m_{1}^{\ast}\left(O,h^{\ast}\right)\right\}$ is linear and it is continuous when $E_{\eta}\left\{P_{\eta}\left(D=1|Z\right)^{-1}\right\}<\infty$ and has Riesz representer $\mathcal{R}_{1}^{\ast}\left(Z^{\ast}\right)=D/E_{\eta}\left(D|Z^{\ast}\right).$ Thus, under the latter condition, the parameter falls in the class of Chernozhukov et al., 2018b . Because $a^{\ast}\left(Z^{\ast}\right)$ is a conditional expectation given $Z^{\ast},$ Proposition 3 implies that $\chi\left(\eta\right)$ has the mixed bias property for $a^{\ast}\left(Z^{\ast}\right)$ as defined, $S_{ab}^{\ast}=1$ and $b^{\ast}\left(Z^{\ast}\right)=D/E_{\eta}\left(D|Z\right).$ In the Supplementary Web Appendix we argue that this example implies that the average treatment effect contrast is a difference of two parameters, each belonging to both the class of Robins et al., (2008) and of Chernozhukov et al., 2018b .

Example 2** (Expected conditional covariance)**

Let $O=\left(Y,D,Z\right),$ where $Y$ and $D$ are real valued. Let $\chi\left(\eta\right)\equiv E_{\eta}\left\{cov_{\eta}\left(D,Y|Z\right)\right\}$ be the expected conditional covariance between $D$ and $Y$. When $D$ is a binary treatment, $\chi\left(\eta\right)$ is an important component of the variance weighted average treatment effect Robins et al., (2008). We can re-write $\chi\left(\eta\right)=E_{\eta}\left(DY\right)+E_{\eta}\left\{m_{1}\left(O,a\right)\right\}$ where $m_{1}\left(O,h\right)\equiv-Dh,$ $a\left(Z\right)\equiv E_{\eta}\left(Y|Z\right)=-$ $E_{\eta}\left\{q\left(O\right)|Z\right\}/E\left(S_{ab}|Z\right)$ with $q\left(O\right)=Y$ and $S_{ab}=-1.$ Then $\chi\left(\eta\right)$ has the mixed bias property with $a\left(Z\right)$ as defined and $b\left(Z\right)=\mathcal{R}_{1}\left(Z\right)=-E_{\eta}\left(D|L\right)$ the Riesz representer of the map $h\in L_{2}\left(P_{\eta,Z}\right)\rightarrow E_{\eta}\left\{m_{1}\left(O,h\right)\right\}$. Thus, $\chi\left(\eta\right)$ is in Chernozhukov et al., 2018b and in the Robins et al., (2008) classes with $S_{a}=-D,$ $S_{b}=Y$ and $S_{0}=DY.$

The next example gives a parameter that is in the class of Robins et al., (2008) but not in the class of Chernozhukov et al., 2018b .

Example 3** (Mean of an outcome missing not at random)**

Suppose $O=\left(DY,D,Z\right)$ where $D$ is binary, $Y$ is an outcome which is observed if and only if $D=1$ and $Z$ is a vector of always observed covariates. If we make the untestable assumption that the density $p\left(y|D=0,Z\right)$ is a known exponential tilt of the density $p\left(y|D=1,Z\right),$ i.e.

[TABLE]

where $\delta$ is a given constant, then under $P=P_{\eta}$ the mean of $Y$ is $\chi\left(\eta\right)=E_{\eta}\left\{DY+\left(1-D\right)a\left(Z\right)\right\}$ assuming $a\left(Z\right)\equiv E_{\eta}\left\{DY\exp\left(\delta Y\right)|Z\right\}/E_{\eta}\left\{D\exp\left(\delta Y\right)|Z\right\}$ exists. Estimation of $\chi\left(\eta\right)$ under different fixed values of $\delta$ has been proposed in the literature as a way of conducting sensitivity analysis to departures from the missing at random assumption (Scharfstein et al.,, 1999). Under the sole restriction $\left(\ref{tilt}\right)$ the law $P$ of the observed data $O$ is unrestricted, and hence the model for $P$ is non-parametric. If $a\left(Z\right)\in L_{2}\left(P_{\eta,Z}\right)$ and $E_{\eta}\left\{D\exp\left(\delta Y\right)|Z\right\}>0,$ then the parameter $\psi\left(\eta\right)\equiv E_{\eta}\left\{m_{1}\left(O,a\right)\right\}$ with $m_{1}\left(O,h\right)\equiv\left(1-D\right)h$ has the mixed bias property because it satisfies the conditions of Proposition 3 with $q\left(O\right)=DY\exp\left(\delta Y\right),$ $S_{ab}=-D\exp\left(\delta Y\right)$ and Riesz representer $\mathcal{R}_{1}\left(Z\right)=E_{\eta}\left(1-D|Z\right)$ and $b\left(Z\right)\equiv-E_{\eta}\left(1-D|Z\right)/E_{\eta}\left\{D\exp\left(\delta Y\right)|Z\right\}.$ Thus, $\chi\left(\eta\right)$ also satisfies the mixed bias property with $S_{ab},$ $a$ and $b$ as defined . The influence function of $\chi\left(\eta\right)$ was derived in Robins and Rotnitzky, (2001) and was shown to be in the Robins et al., (2008) class in that paper. In the Appendix we show that when $\delta\not=0,$ there exists no linear and continuous map $h^{\ast}\in L_{2}\left(P_{\eta,\left(Z\right)}\right)\rightarrow E_{\eta}\left\{m_{1}^{\ast}\left(O,h^{\ast}\right)\right\},$ such that $\psi\left(\eta\right)=E_{\eta}\left\{m_{1}^{\ast}\left(O,a^{\ast}\right)\right\}$ for $a^{\ast}\left(Z\right)=E_{\eta}\left\{q\left(O\right)|Z\right\}$ and $q\left(O\right)$ some statistic. We also show that there exists no linear and continuous map $h^{\ast}\in L_{2}\left(P_{\eta,\left(D,Z\right)}\right)\rightarrow E_{\eta}\left\{m_{1}^{\ast}\left(O,h^{\ast}\right)\right\},$ such that $\psi\left(\eta\right)=E_{\eta}\left\{m_{1}^{\ast}\left(O,a^{\ast}\right)\right\}$ for $a^{\ast}\left(D,Z\right)=E_{\eta}\left\{q\left(O\right)|D,Z\right\}$ and $q\left(O\right)$ some statistic. This shows that $\psi\left(\eta\right),$ and consequently $\chi\left(\eta\right),$ is not in the class studied in Chernozhukov et al., 2018b .

The next example gives a parameter that is in the class of Chernozhukov et al., 2018b but not in the class of Robins et al., (2008)

Example 4** (Causal effect of a treatment taking values on a continuum)**

Let $O=\left(Y,D,L\right),$ $Z=\left(D,L\right),$where $Y$ and $D$ are real valued, $D$ is a treatment variable taking any value in $\left[0,1\right]$ and $L$ is a covariate vector. Furthermore, let $Y_{d}$  denote the counterfactual outcome under treatment $D=d.$ Assume that $E_{\eta}\left(Y_{d}|L\right)=E_{\eta}\left(Y|D=d,L\right)$. The parameter $\chi\left(\eta\right)\equiv E_{\eta}\left\{m_{1}\left(O,a\right)\right\}$ with $a\left(D,L\right)\equiv E_{\eta}\left(Y|D,L\right),m_{1}\left(O,a\right)\equiv\int_{0}^{1}a\left(u,L\right)w\left(u\right)du$ where $w\left(\cdot\right)$ is a given scalar function satisfying $\int_{0}^{1}w\left(u\right)du=0$ agrees with the treatment effect contrast $\int_{0}^{1}E_{\eta}\left(Y_{u}\right)w\left(u\right)du.$ The map $h\in L_{2}\left(P_{\eta,\left(D,Z\right)}\right)\rightarrow E_{\eta}\left\{m_{1}\left(O,h\right)\right\}$ where $m_{1}\left(O,h\right)\equiv\int_{0}^{1}h\left(u,L\right)w\left(u\right)du$ is continuous if $E_{\eta}\left\{\left\{w\left(D\right)/f\left(D|L\right)\right\}^{2}\right\}<\infty$ with Riesz representer $\mathcal{R}_{1}\left(Z\right)=w\left(D\right)/f\left(D|L\right).$ In such case, the parameter $\chi\left(\eta\right)$ is in the class studied in Chernozhukov et al., 2018b . Thus, by Proposition 3 it has the mixed bias property with $S_{ab}=-1,$ $a$ as defined, and $b\left(Z\right)=\mathcal{R}_{1}\left(Z\right)=w\left(D\right)/f\left(D|L\right)$. However, in the Appendix we show that $\chi\left(\eta\right)$ is not in the class of Robins et al., (2008).

The next example gives a parameter that is in neither the class of Chernozhukov et al., 2018b nor in the class of Robins et al., (2008)

Example 5

The following toy example illustrates that there exist parameters $\chi\left(\eta\right)$ that have the mixed bias property but that are in neither the class of  Chernozhukov et al., 2018b nor in the class of Robins et al., (2008). Let $O=\left(Y_{1},Y_{2},Z\right)$ for $Y_{1}$ and $Y_{2}$ continuous random variables, $Y_{2}>0$ and $Z$ a scalar vector taking any values in $\left[0,1\right]$. The parameter $\chi\left(\eta\right)\equiv\int_{0}^{1}a\left(z\right)dz$ where $a\left(Z\right)\equiv E_{\eta}\left(Y_{1}|Z\right)/E_{\eta}\left(Y_{2}|Z\right)$ can be written as $\chi\left(\eta\right)=E_{\eta}\left\{m_{1}\left(O,a\right)\right\}$ where for any $h\left(z\right),$ $m_{1}\left(O,h\right)\equiv\int_{0}^{1}h\left(z\right)dz$ does not depend on $O.$ The map $h\in L_{2}\left(P_{\eta,Z}\right)\rightarrow E_{\eta}\left\{m_{1}\left(O,h\right)\right\}$ is linear. It is continuous if $E_{\eta}\left\{f\left(Z\right)^{-2}\right\}<\infty$ and has Riesz representer $\mathcal{R}_{1}\left(Z\right)=f\left(Z\right)^{-1}.$ In such case, by proposition 3, $\chi\left(\eta\right)$ satisfies the mixed bias property with $S_{ab}=-Y_{2},$ $a$ as defined and $b\left(Z\right)=\left\{f\left(Z\right)E_{\eta}\left(Y_{2}|Z\right)\right\}^{-1}.$ However, it can be shown that the parameter is in neither the class studied in Chernozhukov et al., 2018b nor in the class proposed in Robins et al., (2008)

5 Final remarks

In § 1 we have argued that parameters with the mixed bias property admit estimators with the ‘rate double robustness’ property. However, the class of parameters with the mixed bias property does not exhaust all parameters that admit rate doubly robust estimators. For instance, consider $\psi\left(\eta\right)=g\left\{\chi\left(\eta\right)\right\}$ for a non-linear continuously differentiable function $g$ and a parameter $\chi\left(\eta\right)$ with the mixed bias property. By Theorem 1, the influence function of $\chi\left(\eta\right)$ is of the form $\left(\ref{eq:IF}\right).$ However, the influence function of $\psi\left(\eta\right)$ is $\psi_{\eta}^{1}=g^{\prime}\left\{\chi\left(\eta\right)\right\}\chi_{\eta}^{1}$   which is not of the form $\left(\ref{eq:IF}\right).$ Thus, by Theorem 2 $\psi\left(\eta\right)$ does not have the mixed bias property. Yet, if $\widetilde{\chi}$ is the rate doubly robust, cross-fitted, one step estimator of § $\ref{sec:setup},$ then by the delta method, $\widetilde{\psi}=g\left(\widetilde{\chi}\right)$ is a rate doubly robust estimator of $\psi\left(\eta\right).$ A characterization of the class of all parameters that admit rate doubly robust estimators remains an open question.

6 Supplementary Material

6.1 Examples

Example 6** (Population average treatment effect)**

Suppose that $O=\left(Y,D,Z\right)$ where $D$ is a binary treatment indicator, $Y$ is an outcome and $Z$ is a baseline covariate vector. Under the assumption of unconfoundedness given $Z,$ the population average treatment effect contrast is $ATE\left(\eta\right)\equiv\chi_{1}\left(\eta\right)-\chi_{2}\left(\eta\right)$ where $\chi_{1}\left(\eta\right)\equiv E_{\eta}\left\{a_{1}\left(Z\right)\right\}$ and $\chi_{2}\left(\eta\right)\equiv E_{\eta}\left\{a_{2}\left(Z\right)\right\}$ with $a_{1}\left(Z\right)\equiv E_{\eta}\left(DY|Z\right)/E_{\eta}\left(D|Z\right)$ and $a_{2}\left(Z\right)\equiv E_{\eta}\left\{\left(1-D\right)Y|Z\right\}/E_{\eta}\left\{\left(1-D\right)|Z\right\}$. Regarding 1-D\,\as another missing data indicator, example $\left(\ref{ex:MAR}\right)$ implies that $ATE\left(\eta\right)$ is a difference of two parameters, $\chi_{1}\left(\eta\right)$ and $\chi_{2}\left(\eta\right),$ each in the class of Robins et al., (2008) and of Chernozhukov et al., 2018b .

Example 7** (Mean of outcome missing at random in the non-respondents)**

With the notation and assumptions of Example 1, $E_{\eta}\left\{\left(1-D\right)a\left(Z\right)\right\}/E_{\eta}\left(1-D\right)$ where again, $a\left(Z\right)\equiv E_{\eta}\left(DY|Z\right)/E_{\eta}\left(D|Z\right),$ is equal to the mean of $Y$ among the non-respondents, i.e. in the population with $D=0.$ If $a\left(Z\right)\in L_{2}\left(P_{\eta,Z}\right)$ and $E_{\eta}\left(D|Z\right)>0,$ then the parameter $\chi\left(\eta\right)\equiv$ $E_{\eta}\left\{\left(1-D\right)a\left(Z\right)\right\}$ satisfies the conditions of  Proposition 3 with $m_{1}\left(O,h\right)\equiv\left(1-D\right)h$ and $S_{0}=0.$ The map $h\in L_{2}\left(P_{\eta,Z}\right)\rightarrow E_{\eta}\left\{m_{1}\left(O,h\right)\right\}$ is continuous with Riesz representer $\mathcal{R}_{1}\left(Z\right)=E_{\eta}\left\{\left(1-D\right)|Z\right\},$ and $a\left(Z\right)=-E_{\eta}\left\{q\left(O\right)|Z\right\}/E_{\eta}\left(S_{ab}|Z\right)$ for $S_{ab}=-D$ and $q\left(O\right)=DY.$ Consequently, $\chi\left(\eta\right)$ has the mixed bias property for $a\left(Z\right)$ as defined and $b\left(Z\right)=E_{\eta}\left\{\left(1-D\right)|Z\right\}/E_{\eta}\left(D|Z\right).$ Since $m_{1}\left(O,a\right)=\left(1-D\right)a$, Proposition 2 implies that $\chi\left(\eta\right)$ is in the class of parameters considered by Robins et al., (2008). As in example 1, the parameter $\chi\left(\eta\right)$ is also in the class of Chernozhukov et al., 2018b , for the different ‘covariate’ $Z^{\ast}\equiv$ $\left(D,Z\right)$ and $a^{\ast}\left(Z^{\ast}\right)\equiv E_{\eta}\left(DY|Z^{\ast}\right)$ since we can re-express $\chi\left(\eta\right)$ as $\chi\left(\eta\right)=E_{\eta}\left\{m_{1}^{\ast}\left(O,a^{\ast}\right)\right\}$ where for any $h^{\ast}\left(D,Z\right),$ $m_{1}^{\ast}\left(O,h^{\ast}\right)\equiv\left(1-D\right)h^{\ast}\left(D=1,Z\right).$ The map $h^{\ast}\in L_{2}\left(P_{\eta,\left(D,Z\right)}\right)\rightarrow E_{\eta}\left\{m_{1}^{\ast}\left(O,h^{\ast}\right)\right\}$ is linear and it is continuous when $E_{\eta}\left\{P_{\eta}\left(D=1|Z\right)^{-1}\right\}<\infty$ and has Riesz representer $\mathcal{R}_{1}^{\ast}\left(Z^{\ast}\right)=DE_{\eta}\left\{\left(1-D\right)|Z\right\}/E_{\eta}\left(D|Z\right).$ Thus, under the latter condition, the parameter falls in the class of Chernozhukov et al., 2018b . Because $a^{\ast}\left(Z^{\ast}\right)$ is a conditional expectation given $Z^{\ast},$ Proposition 3 implies that $\chi\left(\eta\right)$ has the mixed bias property for $a^{\ast}\left(Z^{\ast}\right)$ as defined, $S_{ab}^{\ast}=1$ and $b^{\ast}\left(Z^{\ast}\right)=DE_{\eta}\left\{\left(1-D\right)|Z\right\}/E_{\eta}\left(D|Z\right).$

Example 8** (Treatment effect on the treated)**

With the notation and assumptions of Example 6 of the main text, the parameter $ATT\left(\eta\right)\equiv E\left(Y|D=1\right)-\chi\left(\eta\right)/E_{\eta}\left(D\right)$ where $\chi\left(\eta\right)\equiv E_{\eta}\left\{Da\left(Z\right)\right\}$ and $a\left(Z\right)$ defined as $E_{\eta}\left\{\left(1-D\right)Y|Z\right\}/E_{\eta}\left\{\left(1-D\right)|Z\right\}$ the parameter $ATT\left(\eta\right)$ is the average treatment effect on the treated. Once again, regarding $1-D$ as another missing data indicator, Example 7 implies that $ATT\left(\eta\right)$ is a continuous function of a parameter $\chi\left(\eta\right)$ in the class of Robins et al., (2008) and of Chernozhukov et al., 2018b , and other parameters $E\left(Y|D=1\right)$ and $E_{\eta}\left(D\right)$ whose estimation does not require the estimation of high dimensional nuisance parameters

Example 9** (Average policy effect of a counterfactual change of

covariate values)**

Let $\chi\left(\eta\right)\equiv\psi\left(\eta\right)-E_{\eta}\left(Y\right)$ where $\psi\left(\eta\right)=E_{\eta}\left\{a\left(t\left(D\right),L\right)\right\}$ with $a\left(D,L\right)\equiv E_{\eta}\left(Y|D,L\right).$ Then, with the notation and assumptions of example 4 of the main text, $\chi\left(\eta\right)$ is the average policy effect of a counterfactual change $d\rightarrow t\left(d\right)$ of treatment values (Stock, (1989)). Note that $\psi\left(\eta\right)=E_{\eta}\left\{m_{1}\left(O,a\right)\right\}$ where for any $h\left(D,L\right)$ $m_{1}\left(O,h\right)=h\left\{t\left(D\right),L\right\}.$ The functional $h\in L_{2}\left(P_{\eta,\left(D,Z\right)}\right)\rightarrow E_{\eta}\left\{m_{1}\left(O,h\right)\right\}$ is continuous if $E_{\eta}\left[\left\{f_{t}\left(D|L\right)/f\left(D|L\right)\right\}^{2}\right]<\infty$ where $f_{t}\left(D|L\right)$ is the density of $t\left(D\right)$ given $L.$ The Riesz representer of the map is $\mathcal{R}_{1}\left(Z\right)=f_{t}\left(D|L\right)/f\left(D|L\right).$ In such case, $\psi\left(\eta\right)$ is in the class studied in Chernozhukov et al., 2018b , and thus $\chi\left(\eta\right)$ has the mixed bias property, with with $S_{ab}=-1,$ $a\left(Z\right)$ as defined, and $b\left(Z\right)=\mathcal{R}_{1}\left(Z\right)=f_{t}\left(D|L\right)/f\left(D|L\right).$ However, it can be shown that $\chi\left(\eta\right)$ is not in the class of Robins et al., (2008).

6.2 Regularity conditions

We now state the regularity conditions invoked in several of the propositions and theorems in the main text. These are mild conditions that are satisfied in all the examples provided in the main text and in this Appendix.

Condition 1

There exists a dense set $H_{a}$ of $L_{2}\left(P_{\eta,Z}\right)$ such that $H_{a}\cap\mathcal{A}\not=\varnothing$, and for each $\eta$ and each $h\in H_{a},$ there exists $\varepsilon\left(\eta,h\right)>0$ such that $a+th$ $\in\mathcal{A}$ if $\left|t\right|<\varepsilon\left(\eta,h\right)$ where $a\left(Z\right)\equiv a\left(Z;\eta\right).$ The same holds replacing $a$ with $b$ and $\mathcal{A}$ with $\mathcal{B}.$ Furthermore $E_{\eta}\left\{\left|S_{ab}b\left(Z\right)h\left(Z\right)\right|\right\}<\infty$ for $h\in H_{a}$ and $E_{\eta}\left\{\left|S_{ab}a\left(Z\right)h\left(Z\right)\right|\right\}<\infty$ for $h\in H_{b}.$ Moreover for all $\eta$, $E_{\eta}\left[\left|S_{ab}a^{\prime}b^{\prime}\right|\right]<\infty$ for all $a^{\prime}\in\mathcal{A}$ and $b^{\prime}\in\mathcal{B}$.

Condition 2

$\chi\left(\eta\right)$* satisfies the mixed bias property and there exists $b^{\prime}\in\mathcal{B}$ such that for all $\eta,$ (i) $b^{\prime}\left(Z\right)\not=0$ a.s.$\left(P_{\eta,Z}\right),$ and (ii) for the map $m_{2}$ defined in the proof of Theorem 1, $E_{\eta}\left\{m_{2}\left(O,b{\acute{}}\right)|Z\right\}+E_{\eta}\left(S_{ab}|Z\right)b{\acute{}}\left(Z\right)a\left(Z\right)$ is in $L_{2}\left(P_{\eta,Z}\right)$ and $m_{2}\left(O,b\right)-m_{2}\left(O,b{\acute{}}\right)/b{\acute{}}\left(Z\right)$ is in $L_{2}\left(P_{\eta}\right)$.*

6.3 Proofs

Proof:

[of Proposition 1] Let $\eta^{\prime}$ be such that $a^{\prime}=a$ and $b^{\prime}=b.$ Without loss of generality consider a local variation independent parameterization $\eta=\left(a,b,\tau\right)$ and a regular parametric submodel $t\rightarrow\eta_{t}=\left(a_{t},b_{t},\tau_{t}\right).$ Then,

[TABLE]

By $\left(\ref{bias1}\right),$

[TABLE]

[TABLE]

and

[TABLE]

Then,

[TABLE]

Consequently,

[TABLE]

Thus, $\chi_{\eta}^{1}+\chi\left(\eta\right)=\chi\left(\eta^{\prime}\right)+\chi_{\eta^{\prime}}^{1}$ which proves the proposition. $\square$

Proof:

[of Theorem 1] Fix $a^{\ast}\in\mathcal{A}$ and $b^{\ast}\in\mathcal{B}$ and define

[TABLE]

[TABLE]

[TABLE]

For any $h\in\mathcal{A}$ we have

[TABLE]

Likewise, by symmetry we have established that

[TABLE]

We will next show that

[TABLE]

To do so, it suffices to show that

(I) $E_{\eta}\left\{S_{ab}ab+m_{1}^{\ast}\left(O,a\right)+m_{2}^{\ast}\left(O,b\right)+S_{0}^{\ast}\right\}=\chi\left(\eta\right)$ and

(II) $\left.\frac{d}{dt}E_{\eta}\left\{S_{ab}a_{t}b_{t}+m_{1}^{\ast}\left(O,a_{t}\right)+m_{2}^{\ast}\left(O,b_{t}\right)+S_{0}\right\}\right|_{t=0}=0$ for any regular submodel $t\rightarrow P_{\eta_{t}}$

To show (I) we write

[TABLE]

which shows (I)

To show (II) we note that by $\left(\ref{eq:affineproof}\right)$

[TABLE]

and the right hand side does not depend on $a_{t}$. Likewise,

[TABLE]

Thus,

[TABLE]

This shows part (II) and thus concludes the proof of $\left(\ref{eq:IFstar}\right).$

Next, take $a^{{\dagger}}\in$ $H_{a}\cap\mathcal{A}$ and $b^{{\dagger}}\in H_{b}\cap\mathcal{B}$ which we know exist by Condition R.1. Also, by Condition R.1 we know that $a^{\ast\ast}\equiv a^{\ast}+\varepsilon a^{{\dagger}}$ $\in\mathcal{A}$ and $b^{\ast\ast}\equiv b^{\ast}+\varepsilon b^{{\dagger}}$ $\in\mathcal{B}$ for an $\varepsilon>0$ sufficiently small. Now, define $S_{0}^{\ast\ast},m_{1}^{\ast\ast}\left(O,a\right)$ and $m_{2}^{\ast\ast}\left(O,b\right)$ like $S_{0}^{\ast},m_{1}^{\ast}\left(O,a\right)$ and $m_{2}^{\ast}\left(O,b\right)$ but using $a^{\ast\ast}$ and $b^{\ast\ast}$ instead of $a^{\ast}$ and $b^{\ast}.$ Then,

[TABLE]

So, combining this equality with $\left(\ref{eq:IFstar}\right)$ we conclude that

[TABLE]

Thus,

[TABLE]

The right hand side depends on b\,\and the data $O,$ but the left hand side depends on $a$ and the data $O$. Thus, we conclude that $m_{1}^{\ast\ast}\left(O,a\right)-m_{1}^{\ast}\left(O,a\right)$ is a statistic $Q_{1}^{{\dagger}}$ that does not depend on $\eta.$

Now, by $\left(\ref{eq:affineproof}\right),$

[TABLE]

Next, let $S_{0}^{{\dagger}},m_{1}^{{\dagger}}\left(O,a\right)$ and $m_{2}^{{\dagger}}\left(O,b\right)$ be defined like $S_{0}^{\ast},m_{1}^{\ast}\left(O,a\right)$ and $m_{2}^{\ast}\left(O,b\right)$ but using $a^{{\dagger}}$ and $b^{{\dagger}}$ instead of $a^{\ast}$ and $b^{\ast}.$ By \left(\ref{eq:affineproof}\right)\,\applied to $m_{1}^{{\dagger}}\left(O,a\right)$ and $a^{{\dagger}}$ instead of $m_{1}^{\ast}\left(O,a\right)$ and $a^{\ast}$ we have that

[TABLE]

Consequently,

[TABLE]

satisfies

[TABLE]

and therefore the map $h\in\mathcal{A}\rightarrow E_{\eta}\left\{m_{1}\left(O,h\right)\right\}$ is linear. In fact, for any $h_{1},h_{2}\in\mathcal{A}$ and constants $\alpha_{1}$ and $\alpha_{2}$ such that $\alpha_{1}h_{1}+\alpha_{2}h_{2}\in\mathcal{A}$, we know that

[TABLE]

is true for all $\eta.$ Then, for all $\eta^{\prime}$

[TABLE]

By assumption the random variable $r\left(O\right)\equiv m_{1}\left(O,\alpha_{1}h_{1}+\alpha_{2}h_{2}\right)-\left\{\alpha_{1}m_{1}\left(O,h_{1}\right)+\alpha_{2}m_{1}\left(O,h_{2}\right)\right\}$ is in $L_{2}\left(P_{\eta}\right).$ The linearity a.s.$\left(P_{\eta}\right)$ of the map $h\in\mathcal{A}\rightarrow m_{1}\left(O,h\right)$ follows from Lemma 1 below which implies that $r\left(O\right)=0$ a.s.$\left(P_{\eta}\right).$

Likewise, we can show that there exists $Q_{2}^{{\dagger}}$ and $m_{2}\left(O,h\right)\equiv m_{2}^{{\dagger}}\left(O,h\right)-Q_{2}^{{\dagger}}/\varepsilon$ such that $h\in\mathcal{B}\rightarrow E_{\eta}\left\{m_{2}\left(O,h\right)\right\}=-E_{\eta}\left(S_{ab}ah\right)$ and the map $h\in\mathcal{B\rightarrow}m_{2}\left(O,h\right)$ is linear. Finally, define $S_{0}=S_{0}^{{\dagger}}+Q_{1}^{{\dagger}}/\varepsilon+Q_{2}^{{\dagger}}/\varepsilon$ and conclude from $\chi_{\eta}^{1}=S_{ab}ab+m_{1}^{{\dagger}}\left(O,a\right)+m_{2}^{{\dagger}}\left(O,b\right)+S_{0}^{{\dagger}}-\chi\left(\eta\right)$ that

[TABLE]

In addition, from $\left(\ref{eq:m1equality}\right)$ and its analogous for $b,$ we have that

[TABLE]

and

[TABLE]

Consequently,

[TABLE]

thus showing $\left(\ref{eq:parameter}\right)$ holds. This concludes the proof of the Theorem. $\square$

Lemma 1

Suppose that $r\left(O\right)$ is in $L_{2}\left(P_{\eta}\right)$ and that for all $\eta{\acute{}},$ $E_{\eta{\acute{}}}\left\{r\left(O\right)\right\}=0.$ Then $r\left(O\right)=0$ a.s.$\left(P_{\eta}\right).$

Proof:

[of Lemma 1] Suppose first that $r\left(O\right)$ is bounded. Then consider the submodel $t\rightarrow p_{t}\left(O\right)=p_{\eta}\left(O\right)\left\{1+tr\left(O\right)\right\}$ . Note that for $t$ sufficiently small, $p_{t}>0$ (by the boundedness of $r\left(O\right)$) and $p_{t}$ integrates to 1 because $E_{\eta{\acute{}}}\left\{r\left(O\right)\right\}=0.$ Then score of the submodel is $r\left(O\right).$ Then, since by assumption the mean $E_{t}\left\{r\left(O\right)\right\}$ of $r\left(O\right)$ under $p_{t}$ satisfies $E_{p_{t}}\left\{r\left(O\right)\right\}=0$ for all $t,$ we have

[TABLE]

Consequently $r\left(O\right)=0$ a.s.$\left(P_{\eta}\right).$ Next, given an arbitrary $r\left(O\right)$ in $L_{2}\left(P_{\eta}\right)$ such that $E_{\eta{\acute{}}}\left\{r\left(O\right)\right\}=0$ for all $\eta{\acute{}},$ consider define $r_{n}\left(O\right)\equiv r\left(O\right)I_{\left(-n,n\right)}\left\{r\left(O\right)\right\}-E_{\eta}\left\{r\left(O\right)I_{\left(-n,n\right)}\left(r\left(O\right)\right)\right\}.$ Then $r_{n}\left(O\right)$ satisfies $E_{\eta}\left\{r_{n}\left(O\right)\right\}=0$ and is bounded. So, $r_{n}\left(O\right)=0$ a.s.$\left(P_{\eta}\right).$ However, $r_{n}\left(O\right)$ converges in $L_{2}\left(P_{\eta}\right)$ to $r\left(O\right)$ so $r\left(O\right)=0$ a.s.$\left(P_{\eta}\right).$ $\square$

Proof:

[of Theorem 2] For any fixed $h\in H_{a},$ and a given $\eta=\left(a,b,\tau\right)$ consider a parametric submodel $t\rightarrow P_{\eta_{t}}$ where $\eta_{t}=\left(a_{t},b,\tau\right)$ with $a_{t}=a+th$ and $\left|t\right|<\varepsilon\left(\eta,h\right)$ as in Condition 1. Then, since $\chi_{\eta}^{1}$ is an influence function of the form $\left(\ref{eq:IF}\right)$ we have

[TABLE]

The continuity of the maps $h\in L_{2}\left(P_{\eta,Z}\right)\rightarrow E_{\eta}\left\{m_{1}\left(O,h\right)\right\}$ and $h\in L_{2}\left(P_{\eta,Z}\right)\rightarrow E_{\eta}\left\{S_{ab}h\left(Z\right)b\left(Z\right)\right\}$ implies that the map $h\in L_{2}\left(P_{\eta,Z}\right)\rightarrow E_{\eta}\left\{S_{ab}hb+m_{1}\left(O,h\right)\right\}$ is continuous. Then, since we have just shown that this map evaluates to 0 at a dense set of $L_{2}\left(P_{\eta,Z}\right),$ it must equal to 0 for all $h\in L_{2}\left(P_{\eta,Z}\right).$ Reasoning analogously, we arrive at the conclusion that

[TABLE]

for all $h\in L_{2}\left(P_{\eta,Z}\right),$ thus showing part (ii) of the Theorem. Next, suppose that $a^{\prime}-a\in L_{2}\left(P_{\eta,Z}\right)$ and $b^{\prime}-b\in L_{2}\left(P_{\eta,Z}\right).$ Then, applying part (ii) we have that $E_{\eta}\left[S_{ab}\left(a^{\prime}-a\right)b+m_{1}\left\{O,\left(a^{\prime}-a\right)\right\}\right]=0$ and $E_{\eta}\left[S_{ab}\left(b^{\prime}-b\right)a+m_{2}\left\{O,\left(b^{\prime}-b\right)\right\}\right]=0$. Consequently,

[TABLE]

thus showing part (i) of the Theorem.

Turn now to the proof of part (iii). Equation $\left(\ref{eq:unbeq1}\right)$ implies that for all $h\in L_{2}\left(P_{\eta,Z}\right),$

[TABLE]

Thus, if $h^{\ast}\left(Z\right)\equiv E_{\eta}\left(S_{ab}|Z\right)a\left(Z\right)+\mathcal{R}_{2}\left(Z\right)$ is in $L_{2}\left(P_{\eta,Z}\right),$ then specializing at $h=h^{\ast}$ the preceding identity we conclude that a.s.$\left(P_{\eta,Z}\right)$

[TABLE]

or equivalently $a\left(Z\right)=-\mathcal{R}_{2}\left(Z\right)/E_{\eta}\left(S_{ab}|Z\right)$ if $E_{\eta}\left(S_{ab}|Z\right)\not=0$ a.s.$\left(P_{\eta,Z}\right)$. The assertion for  $b\left(Z\right)$ is proved analogously.

Next, we prove part (iv). If $b$ is in $L_{2}\left(P_{\eta,Z}\right),$ then specializing $\left(\ref{eq:unbeq1}\right)$ at $h=b$ we obtain

[TABLE]

On the other hand, if $b\notin L_{2}\left(P_{\eta,Z}\right)$ but $\left(1+t\right)b\in\mathcal{B}$ for $0<t<\varepsilon$ then, given $\eta=\left(a,b,\tau\right)$ consider the parametric submodel $t\rightarrow P_{\eta_{t}}$ where $\eta_{t}=\left(a,b_{t},\tau\right)$ with $b_{t}=b+tb$ and $0<t<\varepsilon.$ Then, by $\chi_{\eta}^{1}$ of the form $\left(\ref{eq:IF}\right)$ being an influence function and with $\frac{d}{dt^{+}}$ denoting the right derivative, we have

[TABLE]

So, applying again $\left(\ref{eq:form1}\right)$ we arrive at $\chi\left(\eta\right)=E_{\eta}\left\{m_{1}\left(O,a\right)+S_{0}\right\}.$ The same reasoning, but now taking left derivatives, yields to the same conclusion if $\left(1+t\right)b\in\mathcal{B}$ for $-\varepsilon<t<0.$ This shows (iv.b). Part (iv.a) is proved analogously. Finally, part (iv.c) follows from

[TABLE]

Turn now to the proof of part (v). By part (iii) we have that a.s.$\left(P_{\eta,Z}\right)$

[TABLE]

Next, for any $h\in$ $L_{2}\left(P_{\eta,Z}\right),$ write

[TABLE]

So

[TABLE]

The assertion for the minimization leading to $a\left(Z\right)$ is proved analogously. $\square$

Proof:

[of Proposition 2] With the definition of $m_{2}$ given in the proof of Theorem 1 we have that $E_{\eta}\left\{m_{2}\left(O,h\right)\right\}=-E_{\eta}\left\{S_{ab}ha\right\}$ for all $h\in\mathcal{B}$. Fix $b^{\prime}\in\mathcal{B}$ such that $b^{\prime}\left(Z\right)\not=0$ a.s.$\left(P_{\eta,Z}\right).$ Then,

[TABLE]

Since by assumption $a$ does not depend on $\eta{\acute{}}_{1},$ then $s_{\eta_{2}}\left(Z\right)\equiv E_{\eta_{2}}\left\{m_{2}\left(O,b^{\prime}\right)|Z\right\}+E_{\eta_{2}}\left(S_{ab}|Z\right)b^{\prime}\left(Z\right)a\left(Z\right)$ is a fixed function of $Z$ (i.e. independent of $\eta{\acute{}}_{1})$ with mean zero under any marginal law of $Z.$ Hence, since by condition 2, $s_{\eta_{2}}\left(Z\right)$ is in $L_{2}\left(P_{\eta,Z}\right),$ then by Lemma 1, $s_{\eta_{2}}\left(Z\right)=0$ a.s.$\left(P_{\eta,Z}\right),$ from where we conclude that $a\left(Z\right)=-E_{\eta_{2}}\left\{q\left(O\right)|Z\right\}/E_{\eta_{2}}\left(S_{ab}|Z\right)$ for $q\left(O\right)\equiv m_{2}\left(O,b^{\prime}\right)|/b^{\prime}\left(Z\right).$

Next, write for any $\eta$

[TABLE]

and since in the last display $-q\left(O\right)b+m_{2}\left(O,b\right)$ is a statistic independent of $\eta,$ which, by condition 2, is in $L_{2}\left(P_{\eta,Z}\right)$ and the display holds for all $\eta$ then $-q\left(O\right)b+m_{2}\left(O,b\right)=0$ a.s.$\left(P_{\eta}\right)$ for all $\eta.$ This shows that $m_{2}\left(O,h\right)=q\left(O\right)h.$ $\square$

Proof:

[of Proposition 3] For a regular parametric submodel $t\rightarrow\eta_{t}$ with score $g$ at $t=0$ (with $\eta_{t=0}=\eta)$,

[TABLE]

But,

[TABLE]

where

[TABLE]

Thus,

[TABLE]

But,

[TABLE]

where the last identity follows by definition of $a\left(Z\right).$ The last assertion of the Theorem follows by Theorem 2. $\square$

Proof:

Here we prove that the parameter $\psi\left(\eta\right)$ in Example 3 is not in the class studied in Chernozhukov et al., 2018b .

Let $O=\left(DY,D,Z\right).$ Notice that given $D=0,$ $O$ depends only on $Z.$ Let

[TABLE]

where $a\left(Z\right)\equiv E_{\eta}\left\{DY\exp\left(\delta Y\right)|Z\right\}/E_{\eta}\left\{D\exp\left(\delta Y\right)|Z\right\}$ for $\delta\not=0.$ Suppose that there exists $m_{1}^{\ast}\left(O,\cdot\right)$ such that for each $\eta,$ the map $h\in L_{2}\left(P_{\eta,\left(D,Z\right)}\right)\rightarrow E_{\eta}\left\{m_{1}^{\ast}\left(O,h\right)\right\}$ is continuous and linear and such that

[TABLE]

where $a^{\ast}\left(D,Z\right)=E_{\eta}\left\{\left.q\left(O\right)\right|D,Z\right\}$ for some statistic $q\left(O\right)$.

Without loss of generality we can assume that $q\left(O\right)=Dq^{\ast}\left(Y,Z\right)$. To see this write $q\left(O\right)=Dq^{\ast}\left(Y,Z\right)+\left(1-D\right)q^{\ast\ast}\left(Z\right).$ Then,

[TABLE]

where $a_{1}^{\ast}\left(D,Z\right)\equiv E_{\eta}\left\{\left.Dq^{\ast}\left(Y,Z\right)\right|D,Z\right\}$ and $a_{0}^{\ast}\left(D,Z\right)\equiv\left(1-D\right)q^{\ast\ast}\left(Z\right)$. Then, by the assumed linearity of the map $h\in L_{2}\left(P_{\eta,\left(D,Z\right)}\right)\rightarrow E_{\eta}\left\{m_{1}^{\ast}\left(O,h\right)\right\}$ we can now write

[TABLE]

where $S_{0}^{\ast\ast}=m_{1}^{\ast}\left(O,a_{0}^{\ast}\right)+S_{0}^{\ast}$ is a statistic because $a_{0}^{\ast}\left(D,Z\right)$ does not depend on $\eta$.

So, from now on we will assume a^{\ast}\left(D,Z\right)\equiv E_{\eta}\left\{\left.q\left(O\right)\right|D,Z\right\}\,\where $q\left(O\right)=Dq^{\ast}\left(Y,Z\right)$ for some $q^{\ast}$. Note that $q\left(Y,Z\right)$ depends on $Y,$ for otherwise $a^{\ast}\left(D,Z\right)$ would not depend on $\eta.$

Because $\sigma\left(\eta\right)$ is the same functional as $\psi\left(\eta\right)$ then their unique influence functions $\sigma_{\eta}^{1}$ and $\psi_{\eta}^{1}$ must agree. We shall compute next the influence function $\psi_{\eta}^{1}$ of $\psi\left(\eta\right)$. For any path $t\rightarrow\eta_{t}$ through $\eta_{t=0}=\eta$ with score $g$ we have

[TABLE]

So, we conclude that

[TABLE]

On the other hand, letting $\mathcal{R}_{\eta}^{\ast}\left(D,Z\right)$ be the Riesz representer of the map $h\in L_{2}\left(P_{\eta,\left(D,Z\right)}\right)\rightarrow E_{\eta}\left\{m_{1}^{\ast}\left(O,h\right)\right\},$ we have

[TABLE]

from where we conclude that

[TABLE]

The uniqueness of influence functions $\sigma_{\eta}^{1}$ and $\psi_{\eta}^{1}$ implies that

[TABLE]

Now, taking $\eta$ and $\eta^{\prime}$ that agree on the law of $Y,D|Z,$ but disagree on the marginal of $Z,$ the left hand side agree on these two laws as well as $a^{\ast}$ so, subtracting one from the other we obtain

[TABLE]

Since $q^{\ast}\left(Y,Z\right)$ depends on $Y$ then $\mathcal{R}_{\eta}^{\ast}\left(D=1,Z\right)-\mathcal{R}_{\eta^{\prime}}^{\ast}\left(D=1,Z\right)$ must be equal to 0. So, we conclude that $R_{\eta}^{\ast}\left(D=1,Z\right)$ depends on $\eta$ only through the law of $Y,D|Z.$

Next, for any $h\left(D,Z\right)=Du\left(Z\right),$ we have

[TABLE]

So, since $\mathcal{R}_{\eta}^{\ast}\left(D=1,Z\right)$ does not depend on the marginal law of $Z,$ we conclude that

[TABLE]

or equivalently

[TABLE]

Suppose that $z^{\ast}$ is such that $u\left(z^{\ast}\right)=0.$ Then,

[TABLE]

where to arrive at the left hand side we have used the fact that $E_{\eta}\left\{m_{1}^{\ast}\left(O,h\right)|D=0,Z\right\}=m_{1}^{\ast}\left\{\left(0,0,Z\right),h\right\}$. Now, since $E_{\eta}\left\{m_{1}^{\ast}\left(O,h\right)|D=1,Z=z^{\ast}\right\}$ does not depend on the law of $D|Z,$ then letting $E_{\eta}\left(\left.D\right|Z=z^{\ast}\right)\rightarrow 0$ we conclude that $m_{1}^{\ast}\left\{\left(0,0,z^{\ast}\right),h\right\}=0$ and consequently also $E_{\eta}\left\{m_{1}^{\ast}\left(O,h\right)|D=1,Z=z^{\ast}\right\}=0.$

Next, for any $Z=z$ such that $u\left(z\right)\not=0$ we write

[TABLE]

and since $\mathcal{R}_{\eta}^{\ast}\left(D=1,z\right)$ does not depend on $u$, then taking any other $u^{\ast}$ with $u^{\ast}\left(z\right)\not=0,$ we have

[TABLE]

Once again, since none of the terms in squared brackets depend on the law of $D|Z,$ the right hand side is a linear function of $\alpha\equiv 1/E_{\eta}\left(\left.D\right|Z=z\right)$ which can take any value in $\left(1,\infty\right),$ but the left hand side is identically equal to 0. Therefore,

[TABLE]

So, we conclude that there exists a function $c\left(z\right)$ independent of $\eta$ such that for all $h\left(D,Z\right)=Du\left(Z\right)$

[TABLE]

Next, return to the equations (6.3) and (6.3) and evaluate them at $D=0,$ to obtain

[TABLE]

where $t\left(Z\right)$ is equal to $S_{0}^{\ast}\left(0,0,Z\right),$ i.e. to $S_{0}^{\ast}$ evaluated at $D=0.$ Next, recalling that $a^{\ast}\left(D,Z\right)=DE\left[q^{\ast}\left(Y,Z\right)|D=1,Z\right]$ and invoking $\left(\ref{eq:mainu}\right)$ we conclude that

[TABLE]

where $c\left(z\right)$ and $t\left(z\right)$ are functions of $z$ that do not depend on $\eta$. We will now show that the last equality cannot hold for all $\eta$ if $\delta\not=0.$ To do so, we re-write the last identity as

[TABLE]

If this identity holds for all $\eta,$ then taking expectations on both sides we have that for all $\eta$

[TABLE]

or equivalently, for all $\eta$

[TABLE]

Since the functionals on the left and right hand-sides are identical, their influence functions must agree. Then, taking an arbitrary submodel $t\rightarrow\eta_{t}$ with score $g$ at $\eta_{t=0}=\eta$ we have

[TABLE]

from where we conclude that

[TABLE]

Consequently,

[TABLE]

Invoking the equalities $\left(\ref{eq:id1}\right)$ and $\left(\ref{eq:id2}\right),$ the last identity is the same as

[TABLE]

or equivalently

[TABLE]

The last equality is equivalent to

[TABLE]

The last equation cannot hold for all $\eta.$ To see this, evaluate the left and right and sides at $y$ and $y^{\ast}$ with $y\not=y^{\ast},$ and subtract one from the other, to obtain

[TABLE]

The left hand side depends on $\eta$ whereas the right hand side does not. We will show that this cannot occur when $\delta\not=0$. To do so, let $\eta$ and $\eta^{\prime}$ correspond to two arbitrary distinct laws. Then evaluating the left hand side at $\eta$ and at $\eta^{\prime}$ and subtracting one from the other, we obtain

[TABLE]

This holds for all $y$ and $y^{\ast}$. Then, regarding $y^{\ast},\eta$ and $\eta^{\prime}$ as fixed and $y$ as a free variable the preceding display is of the form

[TABLE]

Next, since $\exp\left(\delta y\right)y$ and $\exp\left(\delta y\right)$ are not the same function of $y,$ then the preceding identity can only hold if $k_{j}\left(D,z\right)=0$ for $j=1,2,3.$ In particular, the equality $k_{j}\left(D,z\right)=0$ implies that

[TABLE]

But since $\eta$ and $\eta^{\prime}$ are arbitrary, this implies that $E_{\eta}\left\{\exp\left(\delta Y\right)|D=1,Z\right\}$ does not depend on $\eta.$ This is a contradiction when $\delta\not=0$ because it would imply that $\exp\left(\delta Y\right)=c^{\prime}\left(Z\right)$ for some function $c^{\prime}.$ This shows that $\sigma\left(\eta\right)$ cannot be equal to $\psi\left(\eta\right).$

Next, we will show that $\psi\left(\eta\right)$ cannot be equal to any parameter of the form

[TABLE]

where $a^{\ast}\left(Z\right)=E_{\eta}\left\{\left.q\left(O\right)\right|Z\right\}$ for some statistic $q\left(O\right),$ the map $h\in L_{2}\left(P_{\eta,Z}\right)\rightarrow E_{\eta}\left\{m_{1}^{\ast}\left(O,h\right)\right\}$ is continuous and linear for each $\eta$ and $S_{0}^{\ast}$ is a statistic. To proceed, just as before, we start by arguing that if the parameter $\kappa\left(\eta\right)$ is the same as $\psi\left(\eta\right)$ then their unique influence functions must agree. We have already computed the influence function $\psi_{\eta}^{1}$ of $\psi\left(\eta\right).$

On the other hand, it is easy to see that the influence function $\kappa_{\eta}^{1}$ of $\kappa\left(\eta\right)$ is

[TABLE]

where $\mathcal{R}_{\eta}^{\ast}\left(Z\right)$ is the Riesz representer of the map $h\in L_{2}\left(P_{\eta,Z}\right)\rightarrow E_{\eta}\left\{m_{1}^{\ast}\left(O,h\right)\right\}.$ Consequently, equating $\kappa_{\eta}^{1}$ with $\psi_{\eta}^{1}$ we obtain

[TABLE]

Now, taking $\eta$ and $\eta^{\prime}$ that agree on the law of $Y,D|Z,$ but disagree on the marginal of $Z,$ the left hand side agree on these two laws as well as $a^{\ast}$ so, subtracting one from the other we obtain

[TABLE]

Since $q\left(O\right)$ depends on $\left(D,Y\right)$ then $\left\{\mathcal{R}_{\eta}^{\ast}\left(Z\right)-\mathcal{R}_{\eta^{\prime}}^{\ast}\left(Z\right)\right\}$ must be equal to 0. So, we conclude that $R_{\eta}^{\ast}\left(Z\right)$ depends on $\eta$ only through the law of $Y,D|Z.$

Next, for any $h\left(Z\right),$ we have

[TABLE]

So, since $\mathcal{R}_{\eta}^{\ast}\left(Z\right)$ does not depend on the marginal law of $Z,$ we conclude that

[TABLE]

The last equality is the same as

[TABLE]

or equivalently

[TABLE]

If $z^{\ast}$ is such that $h\left(z^{\ast}\right)=0,$ then

[TABLE]

and since $E_{\eta}\left\{m_{1}^{\ast}\left(O,h\right)|D=1,Z=z^{\ast}\right\}$ does not depend on the law of $D|Z,$ then since $E_{\eta}\left(D|Z=z^{\ast}\right)$ can take any value in $\left(0,1\right),$ we conclude that $m_{1}^{\ast}\left(0,0,z^{\ast},h\right)=0$.

On the other hand, for $z$ such that $h\left(z\right)\not=0,$ we have

[TABLE]

Consequently, for any other $h^{\ast}$ such that $h\left(z^{\ast}\right)\not=0,$ we have

[TABLE]

Once again, since $E_{\eta}\left\{m_{1}^{\ast}\left(O,h^{\ast}\right)|D=1,Z=z\right\}$ and $E_{\eta}\left\{m_{1}^{\ast}\left(O,h\right)|D=1,Z=z\right\}$ do not depend on the law of $D|Z,$ and since $E_{\eta}\left(D|Z=z\right)$ can take any value in $\left(0,1\right)$ we conclude that

[TABLE]

Consequently, there exists a function $c\left(Z\right)$ such that for all $h$

[TABLE]

Now, evaluating (6.3) and (6.3) at $D=0$

[TABLE]

So, with $t\left(Z\right)\equiv S_{0}^{\ast}\left(0,0,Z\right)$ and with $a^{\ast}\left(Z\right)=E_{\eta}\left\{\left.q\left(O\right)\right|Z\right\}$ substituted for $h$ in $\left(\ref{eq:otra}\right)$ we arrive at the conclusion that the following equality must hold for all $\eta$

[TABLE]

Therefore,

[TABLE]

Again equating the influence functions of the functionals on the right and left hand sides we conclude that

[TABLE]

which leads to a contradiction when $\delta\not=0.$ To arrive at the contradiction we would reason just as we did to show that the equality $\left(\ref{eq:contra}\right)$ leads to a contradiction. $\square$

Proof:

Here we prove that the parameter in Example 4, is in the class of Chernozhukov et al., 2018b but not in the class of Robins et al., (2008).

Let

[TABLE]

Its influence function is

[TABLE]

Suppose the parameter $\chi\left(\eta\right)$ is in the class of Robins et al., (2008) for some functions $a^{\ast}\left(D,L\right)$ and $b^{\ast}\left(D,L\right)$ which are non-constant in $D$ and in $L.$ Then, there would exist statistics $S_{ab},S_{a},S_{b}$ and $S_{0}$ such that

[TABLE]

and such that

[TABLE]

Then, equating the influence functions we arrive at

[TABLE]

The right hand side does not depend on $f_{\eta}\left(D|L\right).$ On the other hand, in the left hand side

[TABLE]

does not depend on $f_{\eta}\left(D|L\right)$ but $\frac{w\left(D\right)}{f_{\eta}\left(D|L\right)}\left\{Y-E_{\eta}\left(Y|D,L\right)\right\}$ depends on $f_{\eta}\left(D|L\right)$. This is a contradiction.

Now, suppose that we re-define $Z=L,$ so that we now partition $O$ into $\left(Y,D\right)$ and $Z=L.$ If the parameter $\chi\left(\eta\right)$ was in the class of Robins et al., (2008) for some functions $a^{\ast}\left(L\right)$ and $b^{\ast}\left(L\right)$ which are non-constant in $L,$ then, there would exist statistics $S_{ab},S_{a},S_{b}$ and $S_{0}$ such that

[TABLE]

and such that

[TABLE]

Then equating the influence functions we would arrive at

[TABLE]

This is also a contradiction because, for each fixed $D=d,L=l$ the right hand side depends on the value of $f_{\eta}\left(d|l\right),$ i.e. on $f_{\eta}\left(D|L\right)$ evaluated at $D=d,L=l,$ but the right hand side does not, since for a given law of $Y|D,L,$ one can always modify $f_{\eta}\left(D|L\right)$ at a point $D=d,L=l$ and still obtain the same values of $E_{\eta}\left(S_{b}|L=l\right),E_{\eta}\left(S_{a}|L=l\right)$ and $E_{\eta}\left(S_{ab}|L=l\right).$ $\square$