Iterative Estimation of Nonparametric Regressions with Continuous Endogenous Variables and Discrete Instruments

TL;DR

This paper introduces an iterative method for estimating nonparametric regression models with continuous endogenous variables using only discrete instruments, addressing a challenging nonlinear integral equation problem.

Contribution

It proposes a simple iterative estimation procedure for such models and analyzes its asymptotic properties, especially with binary instruments.

Findings

The method effectively estimates the regression function.

Asymptotic properties are established.

Application to returns to education demonstrates practical utility.

Abstract

We consider a nonparametric regression model with continuous endogenous independent variables when only discrete instruments are available that are independent of the error term. Although this framework is very relevant for applied research, its implementation is challenging, as the regression function becomes the solution to a nonlinear integral equation. We propose a simple iterative procedure to estimate such models and showcase some of its asymptotic properties. In a simulation experiment, we detail its implementation in the case when the instrumental variable is binary. We conclude with an empirical application to returns to education.

Tables

Table 1. Table 1. Summary statistics UK FES.

	Mean	Median	St.Dev	Min	Max
Budget share food	0.2074	0.1959	0.0971	0.0014	0.6867
Log Total Expenditure	5.4215	5.4019	0.4494	3.6090	7.4287
Log Gross Income	5.8581	5.8568	0.5381	2.1972	8.0893
Children	0.6205	1.0000	0.4854	0.0000	1.0000

Equations

$$Y=\phi (Z)+U,E(U)=0,$$

$$E(Y|W)=E(\phi (Z)|W).$$

$$E(\phi (Z)|W)\stackrel{a.s.}{=}0\Rightarrow \phi \stackrel{a.s.}{=}0,$$

$$Y=g_1(Z,\epsilon ),$$

$$Y=\phi (Z)+U,\phi \in L_Z^2.$$

$$MI$$

$$E(\phi (Z)|W)=E(Y|W),$$

$$K\phi =r$$

$$FI$$

$$Y=$$

$$Z=$$

$$\epsilon ,V\text{upmodels}$$

$$F(y,z|w)=\frac{\partial }{\partial z_1}\cdots \frac{\partial }{\partial z_p}P(Y\le y,Z\le z|W=w),$$

$$F(y,z)=\frac{\partial }{\partial z_1}\cdots \frac{\partial }{\partial z_p}P(Y\le y,Z\le z).$$

$$P(U\le u|W=w)=P(U\le u)\forall u,w,$$

$$\int F(\phi (z)+u,z|w)dz=\int F(\phi (z)+u,z)dz.$$

$$T({\phi }_{†})=0,$$

$$T_{\phi }^{\prime }(\tilde{\phi })=\int \left(f(\phi (z)+u,z|w)-f(\phi (z)+u,z)\right)\tilde{\phi }(z)dz,$$

$$T_{{\phi }_{†}}^{\prime }(\tilde{\phi })=\frac{\partial }{\partial \alpha }T({\phi }_{†}+\alpha (\tilde{\phi }-{\phi }_{†}){)}_{|\alpha =0}.$$

$$\mathcal{E}=\{\phi \in L_Z^2:E(\phi )=0\}.$$

$$E(\phi (Z)|U=u,W=w)\stackrel{a.s.}{=}E(\phi (Z)|U=u)\Rightarrow \phi \stackrel{a.s.}{=}0.$$

$$\forall a(Z,U)\in L_{U\times Z}^{2}E(a|U,W)=0\Rightarrow a=0.$$

$$g_{\delta ,\tilde{\phi }}(u,z|w)=f_{Y,Z|W}(\phi (z)+\delta \tilde{\phi }(z)+u,z|w),$$

$$\forall \tilde{\phi }\in \mathcal{E}{\int }_0^1\left\{E_{\delta ,\tilde{\phi }}(\tilde{\phi }|U=u,W=w)-E_{\delta ,\tilde{\phi }}(\tilde{\phi }|U=u)\right\}d\delta =0\Rightarrow \tilde{\phi }=0$$

$${\hat{\phi }}_{j+1}={\hat{\phi }}_j-c{\hat{T}}_{{\hat{\phi }}_j}^{\prime \ast }(\hat{T}({\hat{\phi }}_j)),$$

$$\int \int$$

$$=$$

$$\int \tilde{\phi }(z)\psi (u,w)\left(f_{Y,Z|W}(\phi (z)+u,z|w)-f_{Y,Z}(\phi (z)+u,z)\right)f_U(u)f_W(w)dzdwdu.$$

$$\left(T_{\phi }^{\prime \ast }\psi \right)(z)=\int \int \psi (u,w)\left[\frac{f_{Y,Z,W}(\phi (z)+u,z,w)-f_{Y,Z}(\phi (z)+u,z)f_W(w)}{f_Z(z)}\right]f_U(u)dudw,$$

$$\left(T_{\phi }^{\prime \ast }\psi \right)(z)=E\left[\left(\psi (u,w)-E_W\psi (u,W)\right)f_U(u)|Z=z\right],$$

$${\hat{u}}_{ji}=y_i-{\hat{\phi }}_j(z_i)-\left(\frac{1}{n}\sum \left(y_i-{\hat{\phi }}_j(z_i)\right))\right).$$

$$\hat{T}({\hat{\phi }}_j)(u,w)=\frac{\sum _{i=1}^n{\bar{C}}_{h_u}\left(u-{\hat{u}}_{ji}\right)C_{h_w}\left(w-w_i\right)}{\sum _{i=1}^n{C}_{h_w}\left(w-w_i\right)}-\frac{1}{n}\sum _{i=1}^n{\bar{C}}_{h_u}\left(u-{\hat{u}}_{ji}\right),$$

Full text

JEL Codes: C01; C14; C18; C26.

Keywords: Nonparametric; Instrumental Variables; Landweber-Fridman; Engel curve.

Authors would like to thank Enno Mammen for helpful preliminary conversations on this topic. Samuele Centorrino gratefully acknowledges support from the Stony Brook University Initiative for the Arts, Humanities, and Lettered Social Sciences.

1. Introduction

Instrumental variables are a workhorse of applied research in economics. The instrumental variable approach is based on two main assumptions that instruments need to satisfy: exogeneity, i.e. instruments are uncorrelated with or otherwise independent of the structural error term; and relevance, instruments are correlated with or otherwise dependent on the included endogenous regressor. The ability to identify the main causal effect of interest depends on the complexity of the model we want to estimate and on the strength of these two requirements.

In this work, we consider the nonparametric regression model

[TABLE]

where the explanatory variable $Z$ is endogenous. This model is usually identified and estimated by taking the instrumental variable, $W$, to be mean independent of the error term. That is, $E(U|W)=0$, which is an exogeneity restriction on $W$.

Under this relatively weak hypothesis, the regression function $\varphi$ is the solution of the integral equation

[TABLE]

An estimator of $\varphi$ is obtained by solving a regularized version of this functional equation (see Chen and Pouzo, 2012; Darolles et al., 2011; Florens, 2003; Hall and Horowitz, 2005; Horowitz, 2011; Newey and Powell, 2003, among others).

Uniqueness of the solution to equation (2) is usually obtained under the so-called completeness condition, or strong identification (see Florens et al., 1990), which is the relevance assumption in this model. This condition dictates that

[TABLE]

which implies a strong dependence between the endogenous variable, $Z$, and the instrument, $W$. In particular, whenever $Z$ is a continuous variable, completeness requires the instrument $W$ to be itself continuous.111If $W\in\{1,...,L\}$ the set of equations $\int\varphi(Z)f(Z|W=l)dz=0\;\forall l$ cannot characterize a function $\varphi$.

This condition is strong, and may pose some challenges to the empirical application of fully nonparametric estimators of model (1). As a matter of fact, many instruments employed in empirical work are only binary or discrete. This is true, for instance, for randomized experiments with partial compliance, where the intent-to-treat provides an obvious source of exogenous variation (see Krueger, 1999; Torgovitsky, 2015). Card (1995) uses a binary indicator for whether an individual grew up near an accredited college as an instrumental variable to obtain a consistent estimator of returns to schooling. Similarly, Chay and Greenstone (2005) estimate a hedonic model of house prices, using counties’ non-attainment status of US federal pollution threshold as an instrument.

In all these examples, the endogenous variable is continuous (education, pollution, etc), while the instrument is binary, and therefore cannot satisfy the completeness condition as stated above. In general, under standard assumptions of uncorrelated or mean independent instruments, it is not possible to identify any nonlinear, albeit known, function of $Z$ when only a binary instrument is available (Lochner and Moretti, 2015; Løken et al., 2012).

Discrete instruments can be accommodated in this setting either by considering a pseudo-solution to equation (2) (Babii and Florens, 2017b); or by strengthening the exogeneity condition and, as a consequence, weakening the relevance condition. The latter route is the one that we undertake in this paper.

We consider identification and estimation of the model in equation (1) where the instrument, $W$, is taken to be fully independent of the error term $U$. That is, $U\upmodels W$. This strong exogeneity condition leads to characterize $\varphi$ as a solution to a nonlinear integral equation of the first kind (Dunker et al., 2014; Dunker, 2018; Kaltenbacher et al., 2008). In the spirit of Gagliardini and Scaillet (2012), we consider conditions for both local and global identification.

We thoroughly discuss the former concept of identification and derive a condition, termed conditional completeness, which is sufficient to claim identification of the function $\varphi$. We show that this condition is substantially weaker than the usual completeness outlined above. In particular, conditional completeness is satisfied also when $W$ is binary or discrete. Like existing literature, however, we still require that $Z$ and $W$ have no element in common.

We detail the implementation of an iterative Landweber-Fridman algorithm that can be used to estimate the function $\varphi$ under the restriction of full independence, and we derive some of its asymptotic properties.

In a previous paper, Dunker et al. (2014) have considered the estimation and asymptotic properties of nonparametric estimators for nonlinear integral equations. The model analyzed in our paper is presented there as an example (see also Dunker, 2018). The approach in Dunker et al. (2014) is extremely general and oriented to a class of regularization schemes less familiar to econometricians. We believe, however, that the case of discrete instruments is sufficiently important for applied research in econometrics to be separately and carefully developed. We contribute to the existing literature in several directions. First, our discussion of identification is new, and it has not been presented previously. Moreover, the Landweber-Fridman technique that we propose is both computationally efficient and easy to implement. Its properties in this class of statistical problems are new, to the best of our knowledge. Finally, while our approach is not as general as those provided in Dunker et al. (2014) and Dunker (2018), and limited to the statistical model at hand, our convergence rates clearly spell out the sources of the estimation error and allow us to provide better guidance for implementation in this particular example. Ultimately, we believe our paper has both theoretical and empirical value when compared to existing literature.

The model in equation (1), when $U\upmodels W$, is the separable form of a general quantile model with the dependent variable defined by a relation

[TABLE]

with $g_{1}(z,\cdot)$ strictly increasing, $\varepsilon\sim U[0,1]$ and independent of $W$ (see D’Haultfœuille and Février, 2015; Torgovitsky, 2015). Identification and estimation of this model is usually based on a control function assumption à la Imbens and Newey (2009). Our model assumes that the function $g(Z,\varepsilon)$ has a separable form, with $U=F^{-1}_{U}(\varepsilon)$, and $F_{U}$ the cumulative distribution function of $U$. The test of separability in this framework is an important question which has been addressed by Babii and Florens (2017a). In this respect, our approach is less general than control functions that can account for unobserved unidimensional heterogeneity. By contrast, we do not need to specify a triangular structure, i.e. we do not need to specify a first stage equation that relates the endogenous variable to the instrument. More generally, we allow for a direct dependence between the endogenous regressor, $Z$, and the error term, $U$, that is excluded by the control function approach.

The paper is organized as follow. In Section 2, we analyze the identification condition in the independence case in relation with the usual completeness condition. Section 3 discusses the practical implementation of our estimator, whose properties are detailed in Section 4. We discuss the specific case of a binary instrument in Section 5.

We conclude our work with an empirical application to the estimation of the Engel curve for food in Section 6. In this example, the validity of the completeness condition seems to be granted by the normality of the joint distribution of $(Z,W)$, so that the model could be identified under the weaker exogeneity condition (Hoderlein and Holzmann, 2011). We take the instrument to be fully independent of the error term, we consider several discretizations of it and assess their impact on estimation. Our empirical results are robust across different exogeneity conditions, and different discretizations of the instrumental variable.

Notations

In the following, we let $L^{2}$, the space of square integrable functions with respect to the Lebesgue measure. For a real valued function $\varphi\in L^{2}$, we let $\|\varphi\|$ be the $L^{2}$-norm. Finally, for a triplet of random variables $X_{1}$, $X_{2}$ and $X_{3}$, we denote $X_{1}\upmodels X_{2}\|X_{3}$, if $X_{1}$ and $X_{2}$ are independent given $X_{3}$. If $a$ are $b$ are scalar, we let $a\vee b=\max\{a,b\}$, and $a\wedge b=\min\{a,b\}$. For two sequences $a_{n}$ and $b_{n}$, we use the notation $a_{n}\asymp b_{n}$ to signify that the ratio $a_{n}/b_{n}$ is bounded away from zero and infinity.

2. Identification under independence

We consider a random element $\{Y,Z,W\}\in\mathbb{R}\times\mathbb{R}^{p}\times\mathbb{R}^{q}$ and we denote by $L^{2}_{Z}$ or $L^{2}_{W}$ the functions of $Z$ or $W$ square integrable with respect to the true distribution of this random vector. We analyze the model

[TABLE]

This model has been considerably studied under the following mean independence condition

[TABLE]

This condition characterizes $\varphi$ as the solution of the linear equation

[TABLE]

that can also be written as

[TABLE]

where $K:L^{2}_{Z}\rightarrow L^{2}_{W}$, with $r\in L^{2}_{W}$. The solution to (5) is unique (i.e. $\varphi$ is identified) if $K\varphi=0$ implies $\varphi=0$, where all equalities are intended almost surely. Equivalently, $E(\varphi(Z)|W)=0$ implies $\varphi=0$, and the space $L^{2}_{Z}$ does not contain any function orthogonal to $L^{2}_{W}$.

This dependence condition between $Z$ and $W$ goes under the name of $L^{2}$-completeness and has a long history in statistics (see Florens, Mouchart and Rolin (1990) and more recently Andrews (2017), Canay, Santos and Shaikh (2013), D’Haultfoeuille (2011) and Freyberger (2017), among others). This condition is strong: for instance, if $Z$ and $W$ are jointly normal, completeness is equivalent to the equality between the ranks of $Var(Z)$ and $Cov(Z,W)$. If $Var(Z)$ is regular, a necessary condition is then $q\geq p$. If $Z$ is continuous and $W$ discrete, completeness cannot be satisfied.222We notice, however, that if equation (5) is solved using a smooth regularization technique, there exists a solution which converges to an approximation of $\varphi$. See Babii and Florens (2017b). This substantially limits the scope of application of the NPIV model, as in many practical econometric problems, instruments are discrete. This is the case, for instance, for many experimental or quasi-experimental settings with partial compliance, when the reasons for partial compliance are correlated with the output (see, e.g. Krueger, 1999).

We may replace assumption ($MI$) by a stronger assumption of full independence.

[TABLE]

The condition $E(U)=0$, which implies $E(Y)=E(\varphi(Z))$, is a normalization condition. It may be thought to be implicit in the definition of the error term $U$, and, if it is not satisfied, $\varphi$ may be only identified up to an additive constant only. In the following, and without loss of generality, we assume that $E(Y)=E(\varphi(Z))=0$, as the mean of $Y$ can always be identified (provided it exists), and estimated at a parametric rate.

The independence assumption is often used in the following nonseparable triangular model

[TABLE]

where $\varepsilon$ and $V$ follow a uniform distribution in the interval $[0,1]$, and the functions $g_{1}(\cdot,\cdot)$ and $g_{2}(\cdot,\cdot)$ are monotone in their second argument (D’Haultfœuille and Février, 2015; Imbens and Newey, 2009; Torgovitsky, 2015). In our presentation, we focus on the specific properties of the separable model with independent instruments, hence imposing more restrictions on the structural equation. However, we do not need to specify any data generating process for the endogenous variable given the instrument.

It is obvious that ($FI$) implies ($MI$). Then, if the completeness condition between $Z$ and $W$ is verified, $\varphi$ is identified under ($FI$), while the reverse cannot be true.

We therefore want to discuss weaker identification conditions under ($FI$) which may be satisfied in particular in the case where $Z$ is continuous and $W$ discrete.

Let us rewrite the ($FI$) condition. We use the notations

[TABLE]

and

[TABLE]

Roughly speaking $F$ is a cdf as a function of $y$, and a density as a function of $z$.

The condition in ($FI$) is equivalent to

[TABLE]

or

[TABLE]

Equation (9) may be denoted

[TABLE]

and is a non linear integral equation, where $\varphi_{\dagger}$ denotes the solution to this non-linear equation. The functions $F(y,z|w)$ and $F(y,z)$ are identifiable (and estimable) and we have to discuss the solution to equation (9) with respect to the functional parameter $\varphi$.

Let us assume $F(y,z|w)$ and $F(y,z)$ are differentiable w.r.t. $y$. Their first partial derivatives with respect to $y$ are the conditional density and the density, denoted $f_{Y,Z|W}(y,z|w)$ and $f_{Y,Z}(y,z)$. We take these density to be continuous and to have continuous and bounded first partial derivative with respect to their first argument. Under these conditions, $T$ is Fréchet differentiable and $T^{\prime}_{\varphi}(\tilde{\varphi})$ verifies

[TABLE]

where $T^{\prime}_{\varphi}(\tilde{\varphi})$ denotes the derivative of $T$ computed in $\varphi$ as a linear function of $\tilde{\varphi}$.

The non-linear operator $T$ is defined on the zero mean $L^{2}_{Z}$ functions and valued in $L^{2}_{U\times W}$ and the derivative $T^{\prime}$ operates between these spaces.

The derivative $T^{\prime}_{\varphi}(\tilde{\varphi})$ is a linear continuous operator for any $\varphi$ and under minor regularity conditions is an Hilbert Schmidt operator (see Carrasco et al., 2007).

Let us introduce the following definition.

Definition 2.1**.**

The separable NPIV model is linearly identified if the operator $T$ is Fréchet differentiable and $T^{\prime}_{\varphi}$ is a one-to-one linear operator.

Note that this concept is different from the local identification concept introduced in Florens and Sbaï (2010), because we do not assume that $T^{\prime}_{\varphi}$ is continuously invertible. It is equivalent to the concept of local identification for ill-posed inverse problems introduced in Chen et al. (2014). We assume Fréchet differentiability of the operator (see Chen et al., 2014, Assumption 2, p. 789), which helps control the behavior of our nonlinear problem in the vicinity of the true solution. Moreover, we further take the Fréchet derivative to be injective, which is tantamount to a rank condition on $T^{\prime}_{\varphi}$ (see Chen et al., 2014, Assumption 1, p. 788).

We may illustrate this concept by the following argument. Let $\varphi_{\dagger}$ be the true value and $\tilde{\varphi}$ another function. As Fréchet differentiability implies Gâteaux differentiability

[TABLE]

Let us call $A(\alpha)$ the function $T(\varphi_{\dagger}+\alpha(\tilde{\varphi}-\varphi_{\dagger}))$ for $\tilde{\varphi},\varphi_{\dagger}$ fixed, and $\alpha\in\mathbb{R}$. We can apply the implicit function theorem, which implies that the solution $\varphi_{\dagger}$ is locally unique on the line $\varphi_{\dagger}+\alpha(\hat{\varphi}-\varphi_{\dagger})$, or equivalently $\varphi_{\dagger}$ is unique in an interval on this line characterized by $\alpha\in]-\alpha_{0},\alpha_{0}[$. However this property does not imply in general that $\varphi_{\dagger}$ is unique in a neighborhood of $\varphi\in L^{2}_{Z}$. From this remark it seems that Gâteaux differentiability is sufficient for linear identification. However, we also want $T^{\prime}_{\varphi}$ to be a linear operator which is not necessarily satisfied for Gâteaux derivatives but is true for Fréchet derivatives.

Let

[TABLE]

We consider the following assumption.

Assumption 2.1** (Conditional completeness).**

Let $\varphi\in\mathcal{E}$. Then

[TABLE]

In other words, the projection of $\varphi$ under $L^{2}_{U\times W}$ differs from the projection of $\varphi$ under $L^{2}_{U}$ except if $\varphi$ is constant. Moreover, this constant is equal to [math], under the additional assumption $E(U)=0$.

Note that this assumption is weaker than conditional strong identification, as introduced in Florens et al. (1990). Indeed this concept is defined as

[TABLE]

In Assumption 2.1 we only consider on the form $a(Z,U)=\varphi(Z)-E(\varphi(Z)U)$.

However conditional strong identification implies conditional completeness if $Z$ and $U$ are measurably separated, that is $\varphi(Z)=\psi(U)$, implies $\varphi=\psi=cste$, and under $E(\varphi)=0$ (see Florens et al., 1990).

We have the following property.

Proposition 2.1**.**

Under Assumption 2.1, the separable NPIV model is linearly identified.

To see the importance of this condition let us reproduce an example given in Dunker et al. (2014).

Example 1*.*

Let us assume that \left(\begin{array}[]{l}U\\ Z\end{array}\right)|W\sim N\left(\left(\begin{array}[]{l}0\\ 0\end{array}\right),\left(\begin{array}[]{ll}1&\rho(W)\\ \rho(W)&1\end{array}\right)\right) and $W\in\{0,1\}$. If $\rho(0)\neq\rho(1)$, Assumption 2.1 is verified and the model $Y=\varphi(Z)+U$ ($U\upmodels W$ and $E(\varphi(Z))=0$) is linearly identified. The proof of this result is given in Dunker et al. (2014). We generalize this example in Section 5.

Let us now look at global identification condition. This condition is derived from conditions analyzed by Chernozhukov and Hansen (2005) and Fève et al. (2018).

Let $g(u,z|w)\equiv f_{Y,Z|W}(\varphi(z)+u,z|w)$ the density of $U$ and $Z$ given $W$, and $\tilde{\varphi}$ a function of $Z$ and $\delta\in[0,1]$. Let us define

[TABLE]

the density of $U,Z$ given $W$ perturbed by $\delta$ in the direction $\tilde{\varphi}$. We define $g_{\delta,\tilde{\varphi}}(u,z)$ in a similar way, without conditioning on $W=w$.

We consider the following assumption.

Assumption 2.2** (Perturbed conditional completeness).**

[TABLE]

In the previous expression $E_{\delta,\tilde{\varphi}}(\tilde{\varphi}|U=u,W=w)=\int\tilde{\varphi}(z)g_{\delta\tilde{\varphi}}(u|w)du$ where $g_{\delta,\tilde{\varphi}}(z|u,w)=\frac{g_{\delta,\hat{\varphi}}(u,z|w)}{g_{\delta,\hat{\varphi}}(u|w)}$.

We have the following result.

Proposition 2.2**.**

Under Assumption 2.2, the model is globally identified.

Note that even if global identification implies linear identification, Assumption 2.2 does not generally imply Assumption 2.1. Assumption 2.1 only considers the case when $\delta=0$; while Assumption 2.2 is based on an integral with respect to $\delta$.

The condition for global identification is extremely difficult to verify in practice and have no immediate interpretation. In the next section, we show that linear identification is sufficient to derive the asymptotic properties of the estimator.

3. Estimation

Let us recall that we consider a model $Y=\varphi_{\dagger}(Z)+U$, where $Z$ is potentially correlated with $U$, and $\varphi_{\dagger}$ is the true parameter of interest. We consider this model under the assumption that there exists an instrumental variable, $W$, such that $U\upmodels W$ with $E(U)=0$. The (local and global) identification of this model has been presented in Section 2.

We observe an IID sample $\{(Y_{i},Z_{i},W_{i}),i=1,\dots,n\}$ from the joint distribution of the random vector $(Y,Z,W)$, and we take the support of $Z$ and $W$ to be compact. The support of $Z$ is taken to be the interval $[0,1]^{p}$, without loss of generality.

The estimation procedure is based on equation (11) where $F$ is replaced by a nonparametric estimator. Equation (11) is a non-linear integral equation which defines an ill-posed inverse problem.

In the case of a nonlinear equations, we may consider both concepts of global and local ill-posedness. Global ill-posedness implies that the equation has not a unique solution which depends continuously of $F$. This property has been studied by Gagliardini and Scaillet (2012) in a more general case. Our approach is essentially based on local ill-posedness. This property is characterized by the linear approximation at the true value $T^{\prime}_{\varphi}$, which is generally a compact operator. Then $T^{\prime}_{\varphi}$ does not have a continuous inverse and the problem is locally ill-posed.

The ill-posedness of equation (11) requires to use a regularization technique to obtain consistent estimators. Many solutions exist but it is commonly accepted in the mathematical literature that iterative methods are the simplest to implement (see Kaltenbacher et al., 2008, for a general overview). We limit ourselves to the Landweber-Fridman method which is the easiest iterative method. More sophisticated methods may be used, but they do not seem to be relevant for econometric problems, in which the function of interest, $\varphi$, can be usually taken to be very regular.

Let us first describe the numerical algorithm we use and its implementation. We defer the study of the properties of this estimator to Section 4.

The Landweber-Fridman estimator of $\varphi_{\dagger}$ is based on the following recursive relation

[TABLE]

where $j=0,1,2,\dots,N$ is an integer, and $N>0$ denotes the total number of iterations; $\hat{T}(\hat{\varphi}_{j})$ is an estimator of $T(\varphi)$ computed at the point $\hat{\varphi}_{j}$; $\hat{T}^{\prime\ast}_{\hat{\varphi}_{j}}$ is an estimator of the adjoint of $T^{\prime}_{\hat{\varphi}_{j}}$; and $c<1$ is a strictly positive constant that determines the size of the step between consecutive iterations.

We require an additional step for implementation. That is, we first need to derive a closed-form expression for $T^{\prime\ast}_{\varphi}$. Recall that $T^{\prime}_{\varphi}$ is a linear operator from $L^{2}_{Z}$ into $L^{2}_{U\times W}$. Therefore, $T^{\prime\ast}_{\varphi}$ is a linear operator from $L^{2}_{U\times W}$ into $L^{2}_{Z}$ which ought to satisfy the following relation

[TABLE]

These two expressions can be equivalently rewritten as

[TABLE]

From some elementary computations, we get

[TABLE]

which reduces to

[TABLE]

where $E_{W}$ denotes the expectation taken with respect to the marginal distribution of $W$.

Let us now describe the practical implementation of this algorithm.

• •

We select an initial value $\varphi_{0}$. Different choices of the initial conditions are possible. We may take $\varphi_{0}$ equal to a nonparametric estimation of the conditional expectation of $Y$ given $Z$. This is not a consistent estimator if $Z$ is endogenous but in many case the endogeneity bias is not too strong and $E(Y|Z)$ may be a reasonable starting value. Another possible choice is to solve the linear problem $E(Y|W)=E(\varphi(Z)|W)$ for example by a Tikhonov regularization (see Darolles et al., 2011). If $\varphi_{\dagger}$ is identified under the mean independence condition, this solution is a consistent estimator and the iterations will improve this estimation by using a stronger assumption. If $\varphi_{\dagger}$ is under-identified this estimation gives an approximation (see Babii and Florens, 2017b, and our result in Appendix). Finally, one could use a linear or nonlinear parametric instrumental variable estimator.

• •

At each iteration $j\geq 0$, we compute the estimated centered residuals

[TABLE]

$\hat{T}(\hat{\varphi}_{j})$ can be taken to be the difference between the cdf of $\hat{u}_{ji}$ conditional on $W$ and the unconditional cdf. This difference can be estimated using different methods. In this paper, we use a nonparametric kernel estimator

[TABLE]

where $C$ is a nonparametric kernel (e.g., Gaussian, Epanechnikov), $\bar{C}$ its integral, and $h_{u}$ and $h_{w}$ are strictly positive tuning parameters. Alternatively, one could replace the continuous function $\bar{C}\left(\cdot\right)$ with the step function $\hbox{\it 1\hskip-3.0ptI}(\hat{u}_{ji}\leq u)$. Similarly, if $W$ is discrete, the conditional cdf may be computed by sorting with respect to the different (finite) values of $W$.

The density $f_{U}(u)$ is estimated by usual kernel density method, using the fitted residuals $\hat{u}_{ji}$. Finally, $\hat{T}^{\prime\ast}_{\hat{\varphi}_{j}}(\hat{T}(\hat{\varphi}_{j}))$ is computed as

[TABLE]

with tuning parameter $h$. Bandwidth parameters can be chosen by Silverman’s rule-of-thumb or by cross-validation. Proving optimality of this choice is beyond the scope of this paper.

Finally, we need to select a value for the constant $c\in(0,1)$. Notice that $\|T^{\prime\ast}_{\varphi}\|=1$, as $T^{\prime\ast}_{\varphi}$ is essentially a conditional expectation operator. Therefore, any positive value $c<1$ would guarantee convergence of our iteration scheme, i.e. it guarantees that $cT^{\prime\ast}_{\varphi}T^{\prime}_{\varphi}$ is a contraction. Besides this restriction, the particular choice of $c$ does not matter for our purposes and the solution is insensitive to it. To balance precision and computational speed, we usually take $c=0.5$.

• •

The last point is the choice of the stopping rule. This choice is crucial, as the regularization of the ill-posed inverse problem is provided by the stopping rule. It is common in the mathematical literature to adopt the so-called Morozov’s discrepancy principle (see Blanchard et al., 2018; Kaltenbacher et al., 2008; Mozorov, 1967). This principle leads to iterate up to $N_{0}>0$, such that

[TABLE]

where $\delta$ is a noise that is usually known, and $\tau$ is a positive constant, which depends on the properties of the known operator $T$. Heuristically, this rule is based on the idea that we want to reach a targeted variance, while making the bias as small as possible. In Econometric applications, the value of $\delta$ could usually be determined depending on the properties of estimator of $T$. However, the value of $\tau$ is difficult to establish and depends on the unknown underlying properties of the distribution of the error component. Florens et al. (2018) have successfully applied a similar approach to linear ill-posed inverse problem with Landweber-Fridman regularization. In our problem, however, we have an additional estimation error because of a nonparametrically generated regressor (Mammen et al., 2012), which may blow our variance further as $N\rightarrow\infty$. We therefore proceed as follows. We fix a maximum number of iterations, $N_{max}$, based on the asymptotic theory derived below. We then check the empirical norm of $\hat{T}(\hat{\varphi}_{j})$, at each iteration $j=0,1,2,\dots,N_{max}$, and take $N_{0}$ as the iteration where the empirical norm reaches its minimum. Otherwise, we take $N_{0}=N_{max}$. We assess the finite sample properties of this criterion in Section 5.

4. Asymptotic Properties

For nonlinear inverse problems, iteration methods like the one used here would in general not converge globally. We are able to prove local convergence by appropriately restricting the initial condition and controlling the behavior of the Fréchet derivative of the operator $\hat{T}$.

We let $\mathcal{D}(T)$ to be the domain of the operator $T$. We assume the following.

Assumption 4.1**.**

The function $\varphi_{\dagger}$ is identified.

Assumption 4.2**.**

Let $\mathcal{B}_{2R}(\varphi_{0})$ to be a ball of radius $R<\infty$ around the initial condition, such that $\mathcal{B}_{2R}(\varphi_{0})\subset\mathcal{D}(T)$. We have

[TABLE]

Assumption 4.1 assumes identification of the parameter of interest. This issue has been discussed in some length in Section 2.

The condition in Assumption 4.2 is sufficient to guarantee local convergence of the iterative procedure. In practice, large values of $R$ are possible (see also Dunker, 2018).

Second, we need to impose regularity conditions on the linear operator $T^{\prime}_{\varphi_{\dagger}}$ and its adjoint. Our identification conditions imply that the Fréchet derivative is an injective operator, and its definition as a conditional expectation operator directly gives that $\|T^{\prime}_{\varphi_{\dagger}}\|\leq 1$.

We also impose the following.

Assumption 4.3**.**

Let Assumption 4.2 hold. There exists $0<\beta\leq 1$, such that

[TABLE]

with $v\in L^{2}_{Z}$, and $\|v\|<\infty$.

Assumption 4.3 boils down to the so-called Hölder source condition (Engl et al., 2000). This condition is widely used in linear inverse problem to relate the properties of the function of interest with the properties of the conditional expectation operator (see Carrasco et al., 2007, among others). In the nonlinear inverse problem literature a similar condition is imposed (Dunker et al., 2014; Dunker, 2018; Kaltenbacher et al., 2008). However, this condition does not apply directly to the smoothness properties of the true function $\varphi_{\dagger}$, but it is rather a condition on the local properties of our initial guess. In this respect, this assumption does not seem restrictive, and it allows for potentially bad choices of the initial guess.

This restriction for Landweber-Fridman regularization in nonlinear ill-posed inverse problem is not original to us. It is, in fact, well-established in the mathematical literature that, for a source condition as the one in Assumption 4.3, the regularization bias at iteration $j+1$ is of order $(j+1)^{-\beta/2}$. This regularization bias accumulates across $N$ iterations, and, for $\beta$ sufficiently large, it would converge to a non-zero constant as $N\rightarrow\infty$. This could be considered a saturation effect for Landweber-Fridman regularization in the context of nonlinear ill-posed inverse problems.

Assumption 4.3 could be extended to encompass a logarithmic source condition as in Dunker (2018). This is a generalization that we do not pursue here and we defer it to future research.

The conditions imposed so far are usually sufficient to guarantee convergence of any regularization scheme for linear ill-posed inverse problem. For nonlinear inverse problem, however, these conditions are not sufficient to derive an upper bound in $L^{2}$ norm for our estimator. We also need to further restrict the local behavior of the Fréchet derivative. In practice, our implementation of the estimator does not use a local approximation of the operator $T$, but rather an approximation of the adjoint of its Fréchet derivative. We therefore impose here a condition that controls the local behavior of such operator.

It is convenient to recall that, for any functions $\varphi,\tilde{\varphi}\in L^{2}_{Z}$, and $\psi\in L^{2}_{U\times W}$, we can write

[TABLE]

where we take

[TABLE]

to be the kernels of $\hat{T}^{\prime}_{\varphi}$ and $\hat{T}^{\prime\ast}_{\varphi}$, respectively. When $W$ is a discrete variable, integrals can be replaced by sums, where appropriate, and we use the integral notation without loss of generality.

The marginal density of $W$ can be estimated by different methods depending on the nature and the dimension of the instrument. We therefore suppose that there is a function $d(\cdot)$, such that

[TABLE]

This function could be a kernel for continuous or discrete variables (see Aitchison and Aitken, 1976; Li and Racine, 2007); or a product of indicator functions for purely discrete instruments. Then we define

[TABLE]

We choose the points $u$ in expanding sets of the form $\{u:|u|\leq\ell_{n}\}$, where $\ell_{n}$ is a sequence which is either bounded or diverging slowly to infinity (see Hansen, 2008). In this way, we can allow for error distributions with unbounded support.

We assume the following

Assumption 4.4**.**

We have that

[TABLE]

with $\delta_{n},\gamma_{n}\rightarrow 0$, as $n\rightarrow\infty$.

A proof of this assumption under more primitive conditions is given in Section 5, and in the Appendix, for different properties of the instrumental variable, $W$.

We have the following Proposition.

Proposition 4.1**.**

Let Assumptions A.1-A.5 and 4.4 hold, $\tilde{\varphi}\in L^{2}_{Z}$ and $\psi\in L^{2}_{U\times W}$. There exist operators $K_{\varphi_{\dagger},\hat{\varphi}_{j}}$ and $K^{\ast}_{\varphi_{\dagger},\hat{\varphi}_{j}}$ such that

[TABLE]

and

[TABLE]

where $I$ is the identity operator, $R<\infty$, a positive constant, and $\kappa_{n}$ is such that $(h^{2}_{u}\kappa_{n})^{-1}(\delta_{n}\vee\gamma_{n})\rightarrow 0$, as $n\rightarrow\infty$.

A proof of this proposition is in the Appendix. The result of this Proposition effectively shows that we have to pay a price to approximate the density of the unknown error term by the density of the regression residuals. This is tantamount to the usual estimation error with nonparametric generated regressors (Mammen et al., 2012). When the density of the error term is uniformly bounded away from [math] over its support, $\kappa_{n}=R$, a constant. However, as we want to allow for distributions with unbounded support, we let $\kappa_{n}$ converge to [math] as $n$ approaches $\infty$.

Among other things, this proposition implies that

[TABLE]

This condition implies (but it is not implied by) a Lipschitz continuity condition on $\hat{T}^{\prime}$ (see Dunker, 2018; Horowitz and Lee, 2007; Kaltenbacher et al., 2008).

The following Theorem contains the main result of this Section.

Theorem 4.1**.**

Let Assumptions 4.1-4.4, and A.1-A.6 hold. Further let

[TABLE]

and, for $\beta\leq 1/2$,

[TABLE]

Finally, there exists $\beta^{\ast}\in(1/2,1)$, such that

[TABLE]

Then

[TABLE]

where,

[TABLE]

Otherwise, if conditions (20) and (21) do not hold, and we only have $(\delta_{n}\vee\gamma_{n})\sqrt{N\ln(N)}=O(1)$, then

[TABLE]

The result of this Theorem gives an upper bound on the Mean Integrated Squared Error (MISE) of our estimator. Notice that, for $\beta\leq\beta^{\ast}$, as defined in the Theorem, the upper bound is the same one we would get in the context of a linear ill-posed inverse problem with Hölder source condition (Florens et al., 2018). However, for $\beta>\beta^{\ast}$, we cannot reach the same upper bound. Heuristically, this is due to the fact that we have an additional term, $(h_{u}^{2}\kappa_{n})^{-1}$, because of the unknown density of the error term. When $\beta=1$, the regularization bias that accumulates across iterations converges to zero exactly as $N^{-1/2}$, and thus the term $N^{-1/2}/(h_{u}^{2}\kappa_{n})$ dominates. The same effect holds for any $\beta$ close enough to $1$, or, more precisely, for any $\beta>\beta^{\ast}$.

This same heuristic does not apply to the other terms in the decomposition, when we are able to choose $h_{u}$ large enough and $N$ small enough, so that the nonlinearity error does not dominate. Condition (20) on the tuning parameters serves exactly this purpose.

The last statement in the Theorem implies that, if we cannot choose $N$ to grow to $\infty$ slow enough to satisfy condition (20), then we can choose $N$ such that

[TABLE]

which would be equivalent to the optimal choice of the regularization parameter for the linear ill-posed inverse problem (Florens et al., 2018). However, in this case, the rate of convergence would be slower because of the additional term $(h^{2}_{u}\kappa_{n})^{-1}$.

In the next Section, we are going to describe in more details this rate of convergence for the case of a binary instrument.

Example 2*.*

Let us consider the case in which both $Z$ and $W$ are continuous and scalar. We further take kernels of order $\rho\geq 2$ for estimation. In this case, $\hat{T}(\varphi)$ is an estimator of the conditional cdf of $U$ given the instrument $W$, so that one could take

[TABLE]

Let $h_{w}\asymp n^{-\frac{1}{2\rho+1}}$ and $h_{u}\asymp n^{-\frac{1+\epsilon}{2\rho+1}}$, with $\epsilon>0$ small, in a way that $h_{u}=o(h_{w})$ (Li and Racine, 2008). This would imply $\delta_{n}\asymp n^{-\frac{\rho}{2\rho+1}}$. Similarly, $\hat{T}^{\prime}_{\varphi}$ is a conditional expectation operator, and

[TABLE]

where the last equivalence follows by taking $h\asymp n^{-\frac{1}{2\rho+3}}$ (Darolles et al., 2011).Thus, $\delta_{n}\vee\gamma_{n}=\gamma_{n}$.

The condition on the growth of the number of iterations becomes

[TABLE]

which requires $\rho>2$, with $N\rightarrow\infty$, and $\epsilon$ sufficiently small to hold. The result of Theorem 4.1 finally implies

[TABLE]

where the unknown value of $\beta$ determines the optimal divergence of the regularization constant, $N$.

5. Estimation with a binary instrument. Simulations and example

We consider in this section a particular example where $Z\in\mathbbm{R}$ is a continuous variable and $W$ a binary instrument. That is, $W\in\{0,1\}$. This example is similar, and, to some extent, generalizes the one provided in Dunker et al. (2014) and Dunker (2018). We consider the scalar case for simplicity, but all the results can be generalized to any dimensions of the instruments and the endogenous variable.

Recall that, for simplicity, we take $E(Y)=E(\varphi)=0$. Under mean independence, the model is not identified, as the completeness condition fails.

The restriction $E(\varphi|W)=0$ reduces to two conditions

[TABLE]

which cannot imply $\varphi=0$, except when $Z$ is also binary, or when $\varphi(Z)$ is linear in $Z$.

When the completeness condition fails, a smooth regularized estimator of $\varphi$ using the estimated equation $E(Y|W)=E(\varphi(Z)|W)$ converges to the pseudo true value

[TABLE]

where $\lambda_{0}$ and $\lambda_{1}$ are solution to the system

[TABLE]

Then to estimate $\varphi$ using $W$ as an instrument we require additional assumptions, and the assumption $U\upmodels W$ with $E(U)=0$ is convenient for this purpose. We nonetheless need some auxiliary completeness conditions to obtain identification.

Let us consider first the local condition in Assumption 2.1. In this case, it is equivalent to

[TABLE]

Let $f_{U,Z}(u,z|j),j=0,1$ be the conditional density of $\{U,Z\}$ given $W=j$. Let us assume the following decomposition (see Carrasco et al., 2007).

[TABLE]

where the functions $\phi$ and $\psi$ form an orthonormal basis in $L^{2}_{U\times Z}$.

Assumption 2.1 in this context implies that, if the singular value decomposition of the conditional expectation operators is the same when $W$ is equal to [math] or to $1$, then the instrument does not have any identifying power. This is true, whenever $\lambda^{(0)}_{r}=\lambda^{(1)}_{r}$, for all $r=0,1,2,\dots$. On the contrary, whenever $\lambda^{(0)}_{r}\neq\lambda^{(1)}_{r}\quad\forall r\geq 0$, then the condition in Assumption 2.1 is true only if $\varphi=0$. Let

[TABLE]

with $\langle\varphi,\phi_{1}\rangle=0$, as $\varphi$ is assumed to be centered, and

[TABLE]

This equality, along with the condition $E(U)=0$, implies $\varphi=0$, i.e., $\langle\varphi,\phi_{r}\rangle^{2}=0$, for all $r\geq 1$, if and only if $\lambda^{(0)}_{r}\neq\lambda^{(1)}_{r}\quad\forall r\geq 1$, which is exactly our identification condition.

The model is therefore linearly identified if the dependence structure between $Z$ and $U$ captured by the sequence $\lambda^{(j)}_{r}$ is different for $W=1$ and $W=0$.

The global identification condition in Assumption 2.2 becomes in the case where $W\in\{0,1\}$.

[TABLE]

under the further restriction that $E(\tilde{\varphi})=0$.

Let us assume now that

[TABLE]

The condition of Assumption 2.1 becomes

[TABLE]

This property implies $\tilde{\varphi}=0$, as $\lambda^{(1)}_{0}=\lambda^{(0)}_{0}=0$ for any $\delta$ and $\tilde{\varphi}$, such that $E(\tilde{\varphi})=0$.

We now consider the implementation of the estimation method in the particular case of binary instruments.

Let us first look at a parametric approach. Let us assume that $\varphi(z)$ has the form

[TABLE]

where $\theta\in\mathbb{R}^{k}$ and $b$ is given vector of $k$ basis functions (polynomials, splines, etc.). This model is parametric if $k$ is fixed but may be viewed as a sieve estimation if $k$ is allowed to increase with $n$, although we do not pursue this interpretation further. The parametric estimation may be based on the property

[TABLE]

Let us define

[TABLE]

and we estimate $P(U\leq u|W=j)$ by

[TABLE]

where $\bar{C}$ is the cdf of a kernel function of order $\rho\geq 2$, 1I is the indicator function, $h_{u}$ is a bandwidth, and $n_{j}=\sum^{n}_{i=1}\hbox{\it 1\hskip-3.0ptI}(W_{i}=j)$.

We minimize with respect to $\theta$ the Cramér-von Mises distance between the two conditional cdf of $U$ given $W$ which is evaluated by

[TABLE]

for a suitable grid of points $\tilde{u}_{j}$. An example of the application of this method is provided below, and its properties are studied in a companion paper. This approach could also be pursued by replacing cdfs with characteristic functions.

A more flexible method (not dependent on a parametric functional form) follows from the application of the Landweber recursive algorithm described in Section 3.

The only detail that is specific to implementation with a scalar binary instrument is that we do not smooth the conditioning by $W$ but as above we sort the data for $W=1$ or $W=0$.

Let us first look at the asymptotic properties of this estimator. The Fréchet differentiability of the operator $T$ holds as long as the joint pdf of $(Y,Z)$ given $W=w$, is differentiable in its first argument (see also Dunker et al., 2014; Dunker, 2018). This does not appear to be a very strong requirement. We can construct a nonparametric estimator exactly as in equation (24) by replacing the parametric functional form with a known function $\tilde{\varphi}$.

In particular, we have that

[TABLE]

Upon an appropriate choice of the bandwidth parameter, i.e. $h_{u}\asymp n^{-1/2\rho}$, we can take $\delta_{n}\asymp n^{-1/2}$ in Assumption 4.4.

Similarly, as $T^{\prime\ast}_{\varphi}$ is a conditional expectation operator, we can write

[TABLE]

with $\psi\in L^{2}_{U\times W}$, $\hat{f}_{Y,Z,\cdot}(\varphi_{\dagger}(z)+u,z,\cdot)$, $\hat{f}_{Z}(z)$ nonparametric estimators of the densities of observable components of the model, as described in Section 4, with bandwidth parameter $h$; and $\hat{f}_{U}(u)$, nonparametric estimator of the density of the error term, with bandwidth $h_{u}$. As in Darolles et al. (2011), using a second order kernel for the estimation of $\hat{f}_{Y,Z,\cdot}(\varphi_{\dagger}(z)+u,z,\cdot)$ and $\hat{f}_{Z}(z)$, one can prove that

[TABLE]

so that, upon appropriate choice of the bandwidth parameter, i.e. $h\asymp n^{-1/6}$, we can take $\gamma_{n}=n^{-1/3}$ in Assumption 4.4.

For the conditions of the Theorem to be satisfied, we then need

[TABLE]

because $\delta_{n}\vee\gamma_{n}=\gamma_{n}$. Let $\kappa_{n}\asymp n^{-\nu}$, with $\nu>0$, so that

[TABLE]

A sufficient restriction for this condition to hold is

[TABLE]

with $\nu\in(0,1/3)$, so that, to reduce the effect of the unobservability of the error term, it is convenient to take higher order kernels for the estimation of the cdf and pdf of $\hat{U}$.

The second condition given in the Theorem that helps control the bias term is more challenging to interpret, as it depends on two unknown parameters, $\beta$ and $\kappa_{n}$. However, even in this case, we would like to choose $h_{u}$ that goes to [math] as slow as possible, so that we can take $\beta^{\ast}$ as close as possible to $1$. In this respect, using higher order kernel should also help reduce the bias of our estimator.

Otherwise, one could pick the optimal $N$ (denoted here $N_{0}$) to satisfy

[TABLE]

so that

[TABLE]

The latter restriction may be implemented using Morozov’s rule or by extending its modification given in Florens et al. (2018). However, because of the additional term in the upper bound, this rule may induce too much variance in the final estimator.

The bias component in our upper bound depends nonlinearly on $\beta$, and it is therefore more challenging to obtain an optimal order for $N$ based on a squared bias-variance trade-off. We thus propose to select the maximum number of iterations, $N_{max}$, according to equation (26). This rule applies up to a constant, which is inversely proportional to the value of $c$ chosen (the smaller $c$ the higher the number of iterations we need for convergence), and we increase it with the variation in the dependent variable $Y$. In practice, we take this constant to be equal to $\alpha\left(Y_{max}-Y_{min}\right)/c$, with $\alpha>0$. We further take $\rho=8$. As $\nu$ is usually unknown in practice, we take it to be equal to [math] (compact support), and then floor $N_{max}$. For every $N<N_{max}$, we proceed as described in Section 3: we check the difference of the empirical norms of $\hat{T}(\hat{\varphi}_{N})$ and $\hat{T}(\hat{\varphi}_{N-1})$. If this difference is positive (the empirical norm increases from iteration $N-1$ to iteration $N$), we take $N_{0}=N-1$; otherwise we continue. If the difference is negative for every $N\leq N_{max}$, we take $N_{0}=N_{max}$.

The finite sample properties are illustrated via the following simulations. The MatLab code used for this simulation study and the following empirical application is available upon request from the authors.

We take $n=1000$. The instrument $W_{i}\in\{0,1\}$ with probabilities $\left(\frac{1}{3},\frac{2}{3}\right)$ and $U_{i}\sim N(0,1)$. The endogenous variable is generated by the equation

[TABLE]

where $\alpha=1$ and $\beta=2$, $\varepsilon\sim N(0,1)$, and $\sigma=0.5$. The dependent variable is generated as follows

[TABLE]

In both DGPs, the noise-to-signal ratio is about equal to one. We take as starting point for the iterative procedure the nonparametric regression of $Y$ on $Z$. While this estimator is not consistent, it usually correctly approximates the shape of the unknown function and thus provide a reasonable starting value as discussed above. We run $1000$ simulations for each data generating process.

We start by presenting the results for $DGP_{1}$. The left panel of Figure 1 shows the sample, the true curve, the parametric and nonparametric estimators for one sample of simulated data. The dotted blue line represents the plain nonparametric regression estimator, that we use as starting value for our iterative procedure. We only report the parametric estimator using the empirical cdf, as results using the empirical characteristic function do not differ substantially. Our nonparametric estimator behaves reasonably well compared to the fully parametric estimator. On the right panel, we show the decline of the empirical square norm of $\hat{T}(\varphi)$, which reaches its minimum at $N_{0}=810$. In this case, our stopping rule is binding, in the sense that the minimum is reached at the maximum number of iterations. Increasing the number of iterations would have increased the variance of our estimator without necessarily improving the bias as discussed above.

For $DGP_{2}$ we have similar results, although in this case we do not compare our estimator directly with the parametric procedure. The left panel of Figure 2 shows the sample, the true curve, and the nonparametric estimator for one sample of simulated data (where the dotted blue line is again the starting value of our iterative procedure). On the right panel, we show the decline of the empirical square norm of $\hat{T}(\varphi)$, which reaches its minimum at $N_{0}=541$. In this case, our stopping rule is not binding with $N_{max}=660$.

The sampling distribution of both our nonparametric estimates are presented in Figures 3 and 4 where the dotted lines are $95\%$ confidence intervals.

6. An Application to the Estimation of the Engel curve

We now apply our procedure to estimation of the Engel curve for food in a sample of UK households. The goal of this empirical example is twofold: on the one hand, we would like to give a snapshot of the implementation of our approach in a slightly more complicated setting; on the other hand, in this example, the regression function is identified under the completeness condition with a continuous instrument, and we can therefore evaluate how the estimator changes under the full independence assumption and different discretizations of the instrumental variable.

The estimation of the Engel Curve has been used by many authors as a motivating example for studying the properties of nonparametric instrumental regressions (see, e.g. Blundell et al., 2007; Horowitz, 2011, 2014).

The Engel curve describes the expansion path for commodity demands as the household’s budget increases. Therefore, to estimate its shape, it would be sufficient to regress the share of the household’s budget spent for this given commodity, the response variable $Y$, over the total household’s budget, the predictor $Z$. However, the latter is likely to be jointly determined with individual demands, and therefore one ought to consider it as an endogenous regressor.

As discussed in Blundell et al. (2007), the allocation model of income to individual consumption goods and savings suggests exogenous sources of income provide a suitable instrumental variable for total expenditure, as they are likely to be related to the total household expenditure and not to be jointly determined with individual’s budget shares. Consequently, they provide a source of exogenous variations. However, nonlinearities in the total expenditure variable may be required to capture the observed microeconomic behavior in the estimation of the Engel curve (see also Banks et al., 1997; Hausman et al., 1991; Lewbel, 1991), so that a nonparametric specification of the latter seems appropriate.

In the following, we let $Y$ be the budget share for food; $Z$ the natural logarithm of total expenditure; and $W_{0}$ the natural logarithm of gross total income. To reduce heterogeneity in the sample, we also control for the number of children in the family, that we denote $X$. The database is the same as the one used in Blundell et al. (2007), and is obtained from the UK Family Expenditure Survey (FES). It is a subset of married or cohabiting couples with and without children, whose head of household is employed and aged between 20 and 55, and has less than three children. The demographic variable $X$ is thus a dummy variable which takes value $1$ if the couple has one or two children and [math] otherwise. The total sample size is $1655$. Descriptive statistics for these variables are given in Table 1.

Given the nature of our demographic variable, we rewrite our model as follows

[TABLE]

which becomes a varying coefficient model with endogenous functional coefficients (Centorrino and Racine, 2017).

We also generate the following discretizations of the instrumental variable $W_{0}$

[TABLE]

where the thresholds for each category are defined by dividing the support of $W_{0}$ into bins of equal size.

The error term in the model satisfies $U\upmodels W_{j}\|X$, and $X$ uncorrelated with $U$, under full independence; and, $E(U|W_{j},X)=0$, under mean independence, with $j=0,1,2,3$.

When the instrument is continuous, this model can be identified under the completeness condition outlined above (see Blundell et al., 2007; Hoderlein and Holzmann, 2011).

When the instrument is discrete, the completeness condition is not satisfied, although the estimator under mean independence converges to the minimum norm solution (Babii and Florens, 2017b). Our main restriction thus becomes

[TABLE]

We can then implement our estimation procedure exactly as above, by simply conditioning on $X=\{0,1\}$. A more elegant characterization could be obtained following the approach of Centorrino and Racine (2017), and computing the adjoint operator under the varying coefficient restrictions. This goes beyond the scope of the current paper and it is deferred to future research.

When the instrument is binary, we follow the approach outlined in Section 5. When the instrument is either continuous or has multiple categories, we obtain an estimator of the conditional distribution of the error term following the approach of Li and Racine (2008).

Notice that in the latter case the adjoint of the Fréchet derivative is equal to

[TABLE]

All these objects can be estimated exactly as outlined above, conditioning on $X$.

We report in Figures 5 and 6 below the results of our empirical exercise.

We can notice that the nonparametric estimation with full independence is widely consistent across the various discretizations of the instrumental variable. Families with children tend to devote a larger share of their total budget to food, with the share decreasing as the total budget increases. In this sense, there is no loss when using a less precise instrument. We can only conjecture that there may be a larger variance when decreasing the number of categories. To corroborate this point, one would need to develop a consistent bootstrap procedure to construct confidence bands, which we defer for future work.

Similarly, Figure 6 compares the two nonparametric estimators with a continuous instrument. The estimator under the assumption of mean independence is in the left panel, while the one obtained under the assumption of full independence is in the right panel. Ignoring the boundary of the support of $Z$, where fewer observations may be available, the model delivers very similar estimate under different sets of assumptions.

7. Conclusions

We propose in this paper a nonparametric instrumental variable estimator based on an assumption of full independence between the instruments and the structural error term. We discuss conditions for identification, and present an estimator based on a smooth iterative regularization procedure. We derive an upper bound for the Mean Integrated Squared Error and present the finite sample properties in a simulation with one continuous endogenous regressor and a binary instrument. An empirical application to the estimation of the Engel curve demonstrates its features in comparison with an estimator based on a more classical mean independence assumption.

A. Appendix

A.1. Additional Assumptions

To obtain uniform consistency of the nonparametric estimators, we must impose some additional assumptions. These are listed below. Notice that, without loss of generality, we use the word density irrespective of $W$ being discrete or continuous. In the former case, the probability density function can be defined as

[TABLE]

where $p_{W}(w_{k})$ is the probability mass function at $w_{k}$, and $\Delta$ is the Dirac’s delta function, such that

[TABLE]

In many instances below, we also use $R<\infty$ to denote an arbitrary, strictly positive constant.

Assumption A.1**.**

• (i)

The conditional probability density function $f_{Y,Z|W}(y,z|w)$ and the density function $f_{Y,Z}(y,z)$ are $d$ times continuously differentiable and uniformly bounded away from $\infty$.

• (ii)

The densities $f_{Z,W}(z,w)$, $f_{Z}(z)$ and $f_{W}(w)$ are uniformly bounded away from [math] and $\infty$.

Assumption A.2**.**

The density of the error term $f_{U}(u)$ is absolutely continuous with respect to the Lebesgue measure, and $d$ times continuously differentiable.

Assumption A.3**.**

The multivariate kernels $C_{W,h}$ and $C_{Z,h}$ are product kernels generated by the univariate generalized kernel functions $C_{h}$ satisfying the following properties:

• (i)

$C_{h}(\cdot,\cdot)$* is of order $l\geq 2$.*

• (ii)

For each $t\in[0,1]$, the function $C_{h}(h\dot{,}t)$ is supported on $[(t-1)/h,t/h]\cap\mathcal{C}$, where $\mathcal{C}$ is a compact interval that does not depend on t and

[TABLE]

• (iii)

$C_{h}(\cdot,1)=C_{h}(\cdot)$* is a symmetric Lipschitz continuous kernel function with compact support.*

Assumption A.4**.**

Let $\ell_{n}$ a real sequence that is either bounded or diverges slowly to $\infty$ with $n$. The density of the error term, $U$, satisfies

[TABLE]

with $\kappa_{n}\rightarrow 0$, as $n\rightarrow\infty$.

Assumption A.5**.**

The smoothing parameters satisfy $h_{u},h\rightarrow 0$, $(nh_{u})^{-1}\ln n\rightarrow 0$ and $(nh^{p+q+1})^{-1}\ln n\rightarrow 0$.

Assumption A.1 is a standard regularity condition of conditional and unconditional densities. Part $(ii)$ is not restrictive as long as we maintain that the joint support of $(Z,W)$ is compact. Assumption A.2 restricts the density of the error term to be continuous and differentiable. Assumption A.3 defines generalized kernel functions as in Müller (1991). Finally, Assumptions A.4 and A.5 are used for the uniform consistency of the nonparametric density estimators (Hansen, 2008).

A.2. Proofs Section 2

A.2.1. Proof of Proposition 2.1

[TABLE]

As $f(u|w)=f(u),T^{\prime}_{\varphi}(\tilde{\varphi})=0$ is equivalent to $E(\tilde{\varphi}|U,W)=E(\tilde{\varphi}|U)$. Then $\tilde{\varphi}=0$ under Assumption 2.1.

A.2.2. Proof of Proposition 2.2

Let $\varphi$ and $\psi$ two elements of $\mathcal{E}$ such that $T(\varphi)=T(\psi)=0$. Then:

[TABLE]

This equality is equivalent to

[TABLE]

where $\tilde{\varphi}=\psi-\varphi$. If we divide by $g_{\delta,\tilde{\varphi}}(u|w)$ we obtain the condition of Assumption 2.1 which implies $\tilde{\varphi}=0$.

A.3. Additional proofs for Section 4

A.3.1. Proof of Proposition 4.1

Let $\tilde{\varphi}\in L^{2}_{Z}$,

[TABLE]

and

[TABLE]

Then

[TABLE]

Notice that

[TABLE]

where the penultimate line follows from the fact that $U\upmodels W$ implies $Y\upmodels W\|Z$, and the last line from the dependence between $Z$ and $W$ and Assumption A.1(ii). Finally notice that

[TABLE]

so that

[TABLE]

because of Assumption A.1(ii), and Assumption A.4, with the support of the conditional density being a subset of the support of the unconditional density. Thus

[TABLE]

where the penultimate line follows from Assumptions A.1(i), and A.3(iii), with $R(\tilde{R}_{1}+\tilde{R}_{2})<\infty$.

Similarly, we let $\psi\in L^{2}_{U\times W}$

[TABLE]

and

[TABLE]

Thus

[TABLE]

We have

[TABLE]

where the penultimate line follows from Assumptions A.3(iii) and A.4, and the last line from Jensen’s inequality. Finally

[TABLE]

where $\|\cdot\|_{n}$ denotes here the empirical $L^{2}$ norm.

To conclude the proof, notice that

[TABLE]

where $\|\cdot\|_{HS}$ denotes the Hilbert-Schmidt norm of an operator. Similarly for $K^{\ast}_{\varphi_{\dagger},\hat{\varphi}_{j}}$, one can show that

[TABLE]

This completes the proof.

A.3.2. Proof of Theorem 4.1

As the operator $T_{\varphi_{\dagger}}^{\prime\ast}T_{\varphi_{\dagger}}^{\prime}$ is compact and thus admits a singular value decomposition, we use the notation $(T_{\varphi_{\dagger}}^{\prime\ast}T_{\varphi_{\dagger}}^{\prime})^{\beta/2}$ from functional analysis to signify that a function is applied to the singular values of $T_{\varphi_{\dagger}}^{\prime\ast}T_{\varphi_{\dagger}}^{\prime}$. Finally, the source condition implies that we can write $\varphi_{0}-\varphi_{\dagger}=(T_{\varphi_{\dagger}}^{\prime\ast}T^{\prime}_{\varphi_{\dagger}})^{\beta/2}v$, with $v\in L^{2}_{Z}$, and $\|v\|\leq R$.

We first recall the following definition.

Definition A.1** (Qualification).**

A regularization procedure, $g_{N}$, is said to have qualification of order $\kappa>0$, if:

[TABLE]

for $0<\eta\leq\kappa$. ∎

In particular, Landweber-Fridman regularization has qualification equal to $\infty$, in the sense that for every $\eta>0$, the inequality in equation (28) holds with

[TABLE]

Moreover, we need the following.

Assumption A.6**.**

For Landweber-Fridman regularization, there exist two positive constants $R$ and $\eta$ such that:

[TABLE]

∎

Notice that Definition A.1 and Assumption A.6 together imply that

[TABLE]

This result is used repeatedly in the proof below. We have

[TABLE]

where the second line follows from $T(\varphi_{\dagger})=0$. By replacing iteratively $\hat{\varphi}_{j}$, for all $j=0,\dots,N-2$, and letting $\hat{e}_{j}=\hat{\varphi}_{j}-\varphi_{\dagger}$, for all $j=0,1,2,\dots$, we finally obtain

[TABLE]

The first two terms are similar as in the asymptotic expansion of Landweber-Fridman regularization for linear inverse problems, and their bounds are rather standard in this literature (see Florens et al., 2018, for a comparison). By contrast, the terms in $III$ and $IV$ come from the nonlinearity of the inverse problem in our framework. Notice that, as a matter of fact, these latter terms are identically zero when the ill-posed inverse problem is linear. To control these terms, we use the main result provided in Proposition 4.1.

Let $\delta_{n}$ and $\gamma_{n}$ to be defined as in Assumption 4.4. We start by considering the term in $I$. Because of the source condition in Assumption 4.3, we have that

[TABLE]

Under the same conditions outlined earlier, $E||I_{b}||^{2}=O\left(N^{-\beta}\right)$. Moreover, by telescoping the difference of two operators, $A^{N}-B^{N}=\sum_{j=0}^{N-1}A^{N-j-1}(A-B)B^{j}$, we obtain

[TABLE]

where the second and third steps follow from repeated applications of the Minkowski’s inequality. From Lemma A.2, we can bound

[TABLE]

which implies that

[TABLE]

with $\eta<1/2$, and where the last line follows from Lemma A.1, with

[TABLE]

and

[TABLE]

Similarly for $II$, we have

[TABLE]

where the last bound follows from Lemma A.2.

We now control the nonlinear terms. Let

[TABLE]

where the second inequality follows from Lemma A.2 with $s=0.5$. Similarly,

[TABLE]

Notice that, by the latter result, we have

[TABLE]

so that the convergence of the nonlinearity term is dominated by the terms in $IV$.

We prove the result of the Theorem by induction. We let

[TABLE]

and for an integer $l\geq 1$

[TABLE]

The final result is established by controlling the remaining terms. We have

[TABLE]

and we analyze these terms one by one.

First

[TABLE]

Similarly,

[TABLE]

Notice that the condition in equation (26) is enough to control the latter terms so that they are both negligible asymptotically. For the bias component, we have instead

[TABLE]

Thus

[TABLE]

In an analogous fashion, one can prove that

[TABLE]

So that finally,

[TABLE]

Notice that, by the restriction imposed in equation (26), we have that

[TABLE]

Finally, we need to bound the term $h^{-2}_{u}\left(E||III||^{2}\right)^{1/2}$. The bias component can be controlled as follows: for $\beta<1/2$, it is enough to have $(h^{2}_{u}\kappa_{n})^{-1}N^{-\beta/2}=O(1)$, and, for $\beta=1/2$, we need $(h^{2}_{u}\kappa_{n})^{-1}N^{-1/4}\ln(N)=O(1)$. However, for $\beta>1/2$, this requires

[TABLE]

which, for $\beta\leq 1$ and $h^{2}_{u}\kappa_{n}=o(1)$, cannot be satisfied for all $\beta$’s. Therefore, we say there exists a $\beta^{\ast}<1$, such that the condition above is satisfied. Notice that this is equivalent to the condition given in equation (27). Finally, reasoning as above,

[TABLE]

The result of the Theorem follows.

A.4. Proofs for Section 5

We provide here a proof of the partial identification result for the nonparametric instrumental variable estimator under mean independence. We use the example of a continuous endogenous variable and a binary instrument, but the proof can be easily extended to any discrete instrument with a finite number of support points. Recall that $K$ is the conditional expectation operator such that

[TABLE]

and let $\mathcal{N}(K)$ be the null space of $K$. That is,

[TABLE]

Proposition A.1**.**

Let $Z\in\mathbb{R}$ and $W\in\{0,1\}$, such that

[TABLE]

are linearly independent. Thus, the pseudo-true solution to the integral equation

[TABLE]

with $r=E(Y|W)$, is equal to

[TABLE]

where the $\lambda_{j}$’s are the unique solutions to the system of equations

[TABLE]

for $l=\{0,1\}$.

Proof.

Take $\psi\in\mathcal{N}(K)$, and let $\eta_{j}$, for $j=\{0,1\}$, as defined in the proposition. Then, we have that

[TABLE]

This implies that $\eta_{j}\in\mathcal{N}(K)^{\perp}$, the orthogonal space to $\mathcal{N}(K)$. Furthermore, as they are linearly independent they form a basis for $\mathcal{N}(K)^{\perp}$. That is, every element of $\mathcal{N}(K)^{\perp}$ that is orthogonal to $\eta_{0}$ and $\eta_{1}$ is in $\mathcal{N}(K)$, and it is therefore equal to [math]. We conclude that the dimension of $\mathcal{N}(K)^{\perp}$ is $2$ and that every element of $\mathcal{N}(K)^{\perp}$ can be written as a linear combination of $\eta_{0}$ and $\eta_{1}$.

Thus, there must exist constants $\lambda_{0}$ and $\lambda_{1}$ such that, for a $\tilde{\varphi}\in\mathcal{N}(K)^{\perp}$

[TABLE]

Moreover, these $\lambda_{j}$’s must necessarily be unique as, for $\psi=\varphi-\tilde{\varphi}\in\mathcal{N}(K)^{\perp}$, either $K\psi=0$, and $\psi\in\mathcal{N}(K)$, which implies $\varphi=\tilde{\varphi}$; or $K\psi\neq 0$, and thus $\varphi\neq\tilde{\varphi}$. This concludes the proof. ∎

A.5. Useful lemmas

We collect here some lemmas useful for the proofs above. Results are given without proof, and we refer the reader to the corresponding source.

Lemma A.1** (Kaltenbacher et al. (2008, Lemma 2.9, p. 17)).**

Let $a$ and $b$ be non negative. Then there is a positive constant $R(a,b)$ independent of $N$ so that

[TABLE]

with

[TABLE]

Lemma A.2** (Kaltenbacher et al. (2008, Lemma 2.10, p. 18)).**

Let $T$ be a compact operator such that $c\|T\|^{2}\leq 1$, with $T^{\ast}$ be its adjoint. Further let $s\in[0,1]$, and $N\geq 0$, an integer. Then the following estimates hold

[TABLE]