Towards a general theory for non-linear locally stationary processes

TL;DR

This paper develops a comprehensive theoretical framework for non-linear locally stationary processes, establishing fundamental probabilistic results and applying them to nonlinear Markov models and parameter estimation.

Contribution

It introduces a general theory based on stationary approximation and derivatives, extending to nonlinear non-stationary Markov models and asymptotic analysis of maximum likelihood estimates.

Findings

Laws of large numbers and central limit theorems are proved for these processes.

Bias expansions are derived for deterministic and stochastic cases.

Asymptotic properties of MLE in nonlinear non-stationary models are established.

Abstract

In this paper some general theory is presented for locally stationary processes based on the stationary approximation and the stationary derivative. Laws of large numbers, central limit theorems as well as deterministic and stochastic bias expansions are proved for processes obeying an expansion in terms of the stationary approximation and derivative. In addition it is shown that this applies to some general nonlinear non-stationary Markov-models. In addition the results are applied to derive the asymptotic properties of maximum likelihood estimates of parameter curves in such models.

Equations

$$X_{t,n}\approx\tilde{X}_{t}\big{(}\frac{t}{n}\big{)}\approx\tilde{X}_{t}(u_{0})+\Big{(}\frac{t}{n}-u_{0}\Big{)}\,\frac{\partial\tilde{X}_{t}(u)}{\partial u}_{\big{|}u=u_{0}}+\;\mbox{remainder}$$

$$\big{\|}\tilde{X}_{t}(u)-\tilde{X}_{t}(v)\big{\|}_{q}\leq C_{B}|u-v|^{\alpha},\quad\quad\big{\|}X_{t,n}-\tilde{X}_{t}(\frac{t}{n})\big{\|}_{q}\leq C_{B}n^{-\alpha}.$$

$$X_{t,n}=G_{\varepsilon_{t}}\big{(}X_{t-1,n},...,X_{t-p,n},\frac{t}{n}\vee 0\big{)},\quad t\leq n,$$

$$\tilde{X}_{t}(u)=G_{\varepsilon_{t}}\big{(}\tilde{X}_{t-1}(u),...,\tilde{X}_{t-p}(u),u\big{)},\quad t\in\mathbb{Z},$$

$$X_{t,n}=a_{1}\big{(}\frac{t}{n}\big{)}X_{t-1,n}+...+a_{p}\big{(}\frac{t}{n}\big{)}X_{t-p,n}+\sigma\big{(}\frac{t}{n}\big{)}\varepsilon_{t}.$$

$$X_{t,n}=\big{(}a_{0}\big{(}\frac{t}{n}\big{)}+a_{1}\big{(}\frac{t}{n}\big{)}X_{t-1,n}^{2}+...+a_{p}\big{(}\frac{t}{n}\big{)}X_{t-p,n}^{2}\big{)}^{1/2}\varepsilon_{t}.$$

$$X_{t,n}=a_{1}\big{(}\frac{t}{n}\big{)}X_{t-1,n}^{+}+a_{2}\big{(}\frac{t}{n}\big{)}X_{t-1,n}^{-}+\varepsilon_{t},$$

$$X_{t,n}=a_0({\epsilon }_t,\frac{t}{n})+a_1({\epsilon }_t,\frac{t}{n})X_{t-1,n}+...+a_p({\epsilon }_t,\frac{t}{n})X_{t-p,n}.$$

$${\delta }_q^W(k):=\underset{t\in \mathbb{Z}}{\sup }\Vert W_t-W_t^{\ast (t-k)}{\Vert }_q.$$

$${\delta }_q^{\tilde{X}(u)}(k)$$

$$\underset{y\ne y^{\prime }}{\sup }\frac{|g(y)-g(y^{\prime })|}{|y-y^{\prime }{|}_1\cdot (1+|y{|}_1^M+|y^{\prime }{|}_1^M)}\le C,$$

$$\underset{y\ne y^{\prime }}{\sup }\frac{|g(y)-g(y^{\prime })|}{|y-y^{\prime }{|}_1^{\beta }\cdot (1+|y{|}_1^M+|y^{\prime }{|}_1^M)}\le C,$$

$$\frac{1}{n}\sum _{t=1}^n{X}_{t,n}\to {\int }_0^1\mathbb{E}{\tilde{X}}_0(u)\text{mbox}du\text{mbox}in{L}^1$$

$$\frac{1}{nb}\sum_{t=1}^{n}K\Big{(}\frac{t/n-u}{b}\Big{)}\cdot X_{t,n}\to\mathbb{E}\tilde{X}_{0}(u)\quad\mbox{in }L^{1}$$

$$\displaystyle\Big{\|}\sup_{u\in[0,1]}\big{|}\frac{1}{nb}\sum_{t=1}^{n}K\Big{(}\frac{t/n-u}{b}\Big{)}\cdot\big{(}X_{t,n}-\mathbb{E}X_{t,n}\big{)}\big{|}\Big{\|}_{q}$$

$$\le \frac{B_{K}p}{(q-1{)}^2}{\Delta }_{q,0}^{\tilde{X}}\cdot n^{1/q-1}{b}^{-1}+2C_B{B}_K\cdot n^{-\alpha }{b}^{-1}.$$

$$\displaystyle\mathbb{P}\Big{(}\sup_{u\in[0,1]}\big{|}\frac{1}{nb}\sum_{t=1}^{n}K\Big{(}\frac{t/n-u}{b}\Big{)}\cdot\big{(}X_{t,n}-\mathbb{E}X_{t,n}\big{)}\big{|}>x\Big{)}$$

$$\displaystyle\quad\quad\leq\frac{2C_{1}(B_{K}\Delta_{0,q}^{\tilde{X}})^{q}n^{1-q}b^{-q}}{(x/2)^{q}}+8G_{1-2/q}\Big{(}\frac{C_{2}n^{1/2}bx}{2B_{K}\Delta_{0,q}^{\tilde{X}}}\Big{)}+\frac{(2B_{K}C_{B})^{q}}{(x/2)^{q}}\cdot(n^{-\alpha}b^{-1})^{q},$$

$$\underset{u\in [\frac{b}{2},1-\frac{b}{2}]}{\sup }|{\hat{\mu }}_b(u)-\mu (u)|\stackrel{p}{\to }0.$$

$$\{S_{\lfloor nu\rfloor}/\sqrt{n},0\leq u\leq 1\}\overset{d}{\to}\Big{\{}\int_{0}^{u}\sigma(v)\ \mbox{d}B(v),0\leq u\leq 1\Big{\}},$$

$${\sigma }^2(v)=\sum _{k\in \mathbb{Z}}\text{mbox}Cov({\tilde{X}}_0(v),{\tilde{X}}_k(v)).$$

$$\displaystyle\frac{1}{\sqrt{nb}}\sum_{t=1}^{n}K\Big{(}\frac{t/n-u}{b}\Big{)}\cdot\big{\{}X_{t,n}-\mathbb{E}X_{t,n}\big{\}}\overset{d}{\to}N\big{(}0,\int K(x)^{2}\ \mbox{d}x\cdot\sigma^{2}(u)\big{)}$$

$$\tilde{X}_{t}\big{(}\frac{t}{n}\big{)}=\tilde{X}_{t}(u)+\big{(}\frac{t}{n}-u\big{)}\cdot\partial_{u}\tilde{X}_{t}(u)+R_{t,n}\quad a.s.,$$

$$\displaystyle\frac{1}{n}\sum_{t=1}^{n}K_{b}\big{(}\frac{t}{n}-u\big{)}\tilde{X}_{t}(\frac{t}{n})$$

$$\Big{\|}\frac{1}{n}\sum_{t=1}^{n}K_{b}\big{(}\frac{t}{n}-u\big{)}\cdot X_{t,n}-\frac{1}{n}\sum_{t=1}^{n}K_{b}\big{(}\frac{t}{n}-u\big{)}\cdot\tilde{X}_{t}\big{(}\frac{t}{n}\big{)}\Big{\|}_{q}=O(n^{-1}),$$

$$\frac{1}{n}\sum_{t=1}^{n}K_{b}\big{(}\frac{t}{n}-u\big{)}\big{\{}\mathbb{E}X_{t,n}-\mathbb{E}\tilde{X}_{t}(t/n)\big{\}}=O(n^{-\alpha}),$$

$$\frac{1}{n}\sum_{t=1}^{n}K_{b}\big{(}\frac{t}{n}-u\big{)}\big{\{}\mathbb{E}\tilde{X}_{t}(t/n)-\mathbb{E}\tilde{X}_{0}(u)\big{\}}=O(b^{\alpha})+O(n^{-1}).$$

$$\frac{1}{n}\sum_{t=1}^{n}K_{b}\big{(}\frac{t}{n}-u\big{)}\big{\{}\mathbb{E}\tilde{X}_{t}(t/n)-\mathbb{E}\tilde{X}_{0}(u)\big{\}}=o(b)+O(n^{-1}).$$

$$\displaystyle\lim_{h\to 0}\Big{\|}\frac{\tilde{X}_{0}(u+h)-\tilde{X}_{0}(u)}{h}-\tilde{D}_{0}(u)\Big{\|}_{q}$$

$$\tilde{X}_{t}(t/n))=\tilde{X}_{t}(u)+\big{(}\frac{t}{n}-u\big{)}\cdot\partial_{u}\tilde{X}_{t}(u)+\int_{u}^{t/n}\big{\{}\partial_{u}\tilde{X}_{t}(s)-\partial_{u}\tilde{X}_{t}(u)\big{\}}\ \mbox{d}s.$$

$$\big{|}\mathbb{E}\int_{u}^{t/n}\big{\{}\partial_{u}\tilde{X}_{t}(s)-\partial_{u}\tilde{X}_{t}(u)\big{\}}\ \mbox{d}s\big{|}\leq b\cdot\sup_{|u-s|\leq b}\|\partial_{u}\tilde{X}_{t}(s)-\partial_{u}\tilde{X}_{t}(u)\|_{1}=o(b),$$

Full text

Keywords: Non-stationary processes, derivative processes

1 Introduction

One of the challenges in statistics for stochastic processes is always to develop a general theory that goes beyond the investigation of specific models. An elegant example are stationary Gaussian time series or linear time series which are identifiable from the covariance structure or the spectral density of the process. This leads to such powerful tools as the Whittle-likelihood and quite general asymptotic results on statistical inference. This linear theory has been extended to locally stationary processes (cf. Dahlhaus, R. (1997), Dahlhaus, R. (2000),Dahlhaus, R., and Polonik, W. (2009), for an overview see Dahlhaus, R. (2012), Chapter 5) leading again to a general framework in which problems such as bootstrap methods for locally stationary processes (cf. Sergides, M., and Paparoditis, E. (2008),Kreiss, J.P., and Paparoditis, E. (2015)), testing problems (cf. Sergides, M., and Paparoditis, E. (2009),Preuss, P., Vetter, M., and Dette, H. (2013)) long memory models (cf. Palma, W., and Olea, R. (2010), Roueff, F., and Von Sachs, R. (2011)) or dynamic non-stationary factor models (cf. Motta, G., Hafner, C. M., and von Sachs, R. (2011),Eichler, M., Motta, G., and Von Sachs, R. (2011)) can be considered.

In the nonlinear case the situation is more challenging since there is no natural framework similar to the linear Gaussian case. A general theory, however, has been introduced in Wu, W.B. (2005) for Bernoulli shift processes in combination with the functional dependence measure - an important example being Markov processes (and also linear processes). By using this calculus it is possible to transfer a large number of results from the iid case to such processes (for instance M-estimation Wu, W.B. (2007), empirical process theory Wu, W.B. (2008), high-dimensional covariance matrix estimation Chen, X., Xu, M. and Wu, W. B. (2013) - see also the overview in Wu, W.B. (2011)).

The functional dependence measure for Bernoulli shift processes can also be extended to locally stationary processes (cf. Wu, W.B., and Zhou, Z. (2011)) leading to a general framework for non-linear locally stationary processes. Within this framework for example Zhou, Z., and Wu, W.B. (2009) and Wu, Weichi, and Zhou, Z. (2017) discuss quantile regression, Zhou, Z. (2014a) inference for weighted V-statistics and Zhou, Z. (2014b) nonparametric regression for locally stationary processes.

Another general concept for locally stationary processes is the use of stationary approximations and derivative processes introduced in the context of time varying ARCH-processes in Dahlhaus, R., and Subba Rao, S. (2006). The concept has been investigated further in Subba Rao, S. (2006) in the context of random coefficient models. Vogt, M. (2012) uses the stationary approximation for a definition of local stationarity. The concept has also been used for diffusion processes in Koo, B., and Linton, O. (2012).

In this paper the general theory for nonlinear locally stationary processes will be developed further. Our contribution is twofold: First we consider in Sections 2 and 3 processes which admit an expansion in terms of the stationary approximation and the derivative process, and prove several asymptotic results for such processes. We then consider in Section 4 a general Markov-structured locally stationary process and show that it fulfills such an expansion. As a consequence all results of Sections 2 and 3 immediately can be applied for such processes. In addition we use in Section 5 the stationary approximation and the derivative process to derive the asymptotic theory for maximum likelihood estimates.

More precisely we use in Section 2 the stationary approximation to prove global and local laws of large numbers and a central limit theorem which hold under minimal moment assumptions on the process. The proofs make use of the asymptotic theory for sums of stationary sequences. In Section 3 we use the differential calculus connected to derivative processes to derive deterministic and stochastic bias expansions of localized sums. In addition we show accurate error estimations of the Wigner-Ville spectrum and the distribution function of the process.

In Section 4, we consider a class of Markov processes and prove that they satisfy the expansion from Section 3 in terms of the stationary approximation and the derivative process. A difficult part of the proof is the existence of a continuous modification of the stationary approximation and the proof that the derivative process can be obtained as the solution of a functional equation. These results are the prerequisite to apply the functional calculus used in sections 2 and 3 and in section 5 thereafter. We also prove that the functional dependence measure of such processes decays exponentially. Locally stationary Markov processes have recently also been investigated with different type of results in Truquet, L. (2016).

In Section 5, we use these results to investigate nonparametric maximum likelihood estimation of parameter curves in locally stationary processes. Concluding remarks are given in Section 6. Some proofs are postponed to the Supplementary Material 7.

2 General asymptotic results for locally stationary processes

In this and the next section we restrict ourselves to the basic idea of local stationarity, namely that the nonstationary process $X_{t,n}$ can be approximated locally by a stationary process $\tilde{X}_{t}(u)$ in some neighborhood of $u$, that is for those $t$ where $|t/n-u|$ is small. An even better approximation can usually be achieved by using the derivative process $\frac{\partial\tilde{X}_{t}(u)}{\partial u}$ leading heuristically to

[TABLE]

(or higher order approximations by using higher order derivative processes). An example are the processes defined below in (3) and (4) which are investigated in detail in Section 4. In this and the next section we explore what kind of results can be obtained just based on this framework where in this section we just use the stochastic approximation $\tilde{X}_{t}(u)$ (to derive law of large numbers and central limit theorems) while in the next section we use in addition the derivative process $\frac{\partial\tilde{X}_{t}(u)}{\partial u}$ (to derive deterministic and stochastic bias approximations among other results). For a better understanding we make some comments about the situation in advance:

1. In Assumption 2.1(S2) and (S3) we assume that $\tilde{X}_{t}(u)$ is almost surely continuous or continuously differentiable in $u$ respectively. This is a strong assumption since $\tilde{X}_{t}(u)$ is initially defined in most cases pointwise in $u$ as in (4). The existence of continuous or continuously differentiable versions of $\tilde{X}_{t}(u)$ will be proved for (4) in Section 4.

We mention that almost sure differentiability could be replaced by the weaker assumption of differentiability in $L^{q}$ (see Remark 4.3 and Proposition 3.3(b)). The stronger assumption of almost sure differentiability leads to some weaker assumptions in applications (see Remark 3.5) and has the advantage that several results can be obtained in a more straightforward way and that the presentation is simpler.

2. It is obvious that for asymptotic results some form of mixing is needed in addition which we require in Assumption 2.3. We distinguish between mixing conditions on $X_{t,n}$ and the stationary approximation $\tilde{X}_{t}(u)$ since our aim is to show that most of the results can be obtained by only posing mixing assumptions on $\tilde{X}_{t}(u)$ which on the other side leads to an additional approximation error in the results and subsequently in come cases to stronger assumptions.

3. One of the nice features of Assumptions 2.1 and 2.3 is that, provided they hold for the process $X_{t,n}$, they then automatically also hold for a large class of functionals $g(X_{t,n},...,X_{t-r+1,n})$ (under modified moment assumptions). Thus all results for the process $X_{t,n}$ immediately transfer to $g(X_{t,n},...,X_{t-r+1,n})$. This is stated in Proposition 2.5 below.

As usual we are working in the infill asymptotic framework with rescaled time $t/n\in[0,1]$ where $n$ denotes the number of observations. We now assume:

Assumption 2.1** (Stationary approximation).**

Let $q>0$ and $\|W\|_{q}:=(\mathbb{E}|W|^{q})^{1/q}$. Let $X_{t,n}$, $t=1,...,n$ be a triangular array of stochastic processes. For each $u\in[0,1]$, let $\tilde{X}_{t}(u)$ be a stationary and ergodic process such that the following holds.

1. (S1)

$\sup_{u\in[0,1]}\|\tilde{X}_{t}(u)\|_{q}<\infty$. There exists $1\geq\alpha>0,C_{B}>0$ such that uniformly in $t=1,...,n$ and $u,v\in[0,1]$,

[TABLE] 2. (S2)

$u\mapsto\tilde{X}_{t}(u)$* is a.s. continuous for all $t\in\mathbb{Z}$ and $\|\sup_{u\in[0,1]}|\tilde{X}_{t}(u)|\ \|_{q}<\infty$.* 3. (S3)

$\alpha=1$* and $u\mapsto\tilde{X}_{t}(u)$ is a.s. continuously differentiable for all $t\in\mathbb{Z}$ and*

$\|\sup_{u\in[0,1]}|\partial_{u}\tilde{X}_{t}(u)|\ \|_{q}<\infty$.

(S1) allows us to replace $X_{t,n}$ by the stationary approximation $\tilde{X}_{t}(u)$ with rate $|t/n-u|^{\alpha}+n^{-\alpha}$. In many models and statistical applications, $\alpha=1$. In Section 4 (cf. Corollary 4.9) we will show that Assumption 2.1 is fulfilled for example for processes $X_{t,n}$ defined by the recursion

[TABLE]

where $(\varepsilon_{t})_{t\in\mathbb{Z}}$ are i.i.d. random variables, $G:\mathbb{R}\times\mathbb{R}^{p}\times[0,1]\to\mathbb{R}$ and $a\vee b:=\max\{a,b\}$. Here, $\varepsilon_{t}$ take the role of i.i.d. innovations which enter the process in the $t$-th step. For $u\in[0,1]$, the stationary approximation $\tilde{X}_{t}(u)$, $t\in\mathbb{Z}$, in this case is given by the recursion

[TABLE]

(or an a.s. continuous modification). We first give some examples which are covered by our results. These include in particular several classical parametric time series models where the constant parameters have been replaced by time-dependent parameter curves.

Example 2.2**.**

1. (i)

the tvAR($p$) process: Given parameter curves $a_{i},\sigma:[0,1]\to\mathbb{R}$ $(i=1,...,p$),

[TABLE] 2. (ii)

the tvARCH($p$) process (cf. Dahlhaus, R., and Subba Rao, S. (2006)**): Given parameter curves $a_{i}:[0,1]\to\mathbb{R}$ ($i=0,...,p$),

[TABLE] 3. (iii)

the tvTAR($1$) process (cf. Zhou, Z., and Wu, W.B. (2009)**): Given parameter curves $a_{1},a_{2}:[0,1]\to\mathbb{R}$, define

[TABLE]

where $x^{+}:=\max\{x,0\}$ and $x^{-}:=\max\{-x,0\}$. 4. (iv)

the time-varying random coefficient model (cf. Subba Rao, S. (2006)**): With some parameter functions $a_{i}(\cdot)$, $i=0,...,p$,

[TABLE]

We now prove laws of large numbers and a central limit theorem. To specify the necessary mixing conditions we use the uniform functional dependence measure (cf. Liu, W., Xiao, H., and Wu, W. B. (2013)). Let $\varepsilon_{t}$, $t\in\mathbb{Z}$ be a sequence of i.i.d. random variables. For $t\geq 0$, let $\mathcal{F}_{t}:=(\varepsilon_{t},\varepsilon_{t-1},...)$ and $\mathcal{F}_{t}^{*(t-k)}:=(\varepsilon_{t},...,\varepsilon_{t-k+1},\varepsilon_{t-k}^{*},\varepsilon_{t-k-1},\varepsilon_{t-k-2},...)$, where $\varepsilon_{t-k}^{*}$ is a random variable which has the same distribution as $\varepsilon_{1}$ and is independent of all $\varepsilon_{t}$, $t\in\mathbb{Z}$. For a process $W_{t}=H_{t}(\mathcal{F}_{t})\in L^{q}$ with deterministic $H_{t}:\mathbb{R}^{\mathbb{N}}\to\mathbb{R}$ define $W_{t}^{*(t-k)}:=H_{t}(\mathcal{F}_{t}^{*(t-k)})$ and the uniform functional dependence measure

[TABLE]

If $Y_{t}$ is stationary, (5) reduces to the form $\delta^{Y}_{q}(k)=\|Y_{k}-Y_{k}^{*0}\|_{q}$.

Assumption 2.3** (Dependence measure).**

For some $q>0$, assume that

1. (M1)

(dependence measure of the stat. approximation) for each $u\in[0,1]$, there exists a measurable function $H(u,\cdot)$ such that $\tilde{X}_{t}(u)=H(u,\mathcal{F}_{t})$ and $\delta^{\tilde{X}}_{q}(k):=\sup_{u\in[0,1]}\delta_{q}^{\tilde{X}(u)}(k)$ fulfills $\Delta_{0,q}^{\tilde{X}}:=\sum_{k=0}^{\infty}\delta_{q}^{\tilde{X}}(k)<\infty$. 2. (M2)

(dependence measure of the process) for each $t,n\in\mathbb{N}$, there exists a measurable function $H_{t,n}$ such that $X_{t,n}=H_{t,n}(\mathcal{F}_{t})$ with $\Delta_{0,q}^{X}:=\sum_{k=0}^{\infty}\sup_{n\in\mathbb{N}}\delta_{q}^{X_{\cdot,n}}(k)<\infty$. 3. (M3)

$\partial_{u}\tilde{X}_{t}(u)=\partial_{u}H(u,\mathcal{F}_{t})$* and $\delta^{\partial\tilde{X}}_{q}(k):=\sup_{u\in[0,1]}\delta_{q}^{\partial_{u}\tilde{X}(u)}(k)$ is absolutely summable in the sense that $\Delta_{0,q}^{\partial\tilde{X}}:=\sum_{k=0}^{\infty}\delta_{q}^{\partial\tilde{X}}(k)<\infty$.*

In Section 4 we will show that these assumptions are also fulfilled for models which obey (3) and (4). Note that (M1) is a mixing condition on $\tilde{X}_{t}(u)$ (which in most cases is sufficient for asymptotic results) while (M2) is a mixing condition on $X_{t,n}$. In our results, (M1) or (M2) are assumed alternatively. In general our goal is to use mainly assumptions on the stationary approximations, i.e. to use (M1) instead of (M2). This also allows $X_{t,n}$ to have a structure which is different from $H_{t,n}(\mathcal{F}_{t})$, for instance contamination with some random noise which is decreasing in $n$ and is not produced by the innovations $\varepsilon_{i}$. Note that posing only (M1) forces us to replace $X_{t,n}$ in the proofs by its stationary approximation which naturally leads to an approximation error $n^{-\alpha}$. In many practical cases, we have $\alpha=1$. It can be seen in our results that this implies negligibility of this error.

For the model $\tilde{X}_{t}(u)=H(u,\cdot)$, $X_{t,n}=H_{t,n}(\mathcal{F}_{t})$ with (S1), then (M2) implies (M1) while the reverse implication is false. This can be seen as follows: For arbitrary $u\in[0,1]$ choose $t_{n}\in\{1,...,n\}$ with $|u-t_{n}/n|\leq n^{-\alpha}$. Then

[TABLE]

which by $n\to\infty$ implies (M1). If for instance for each fixed $u\in[0,1]$, $\tilde{X}_{t}(u)$ is a simple AR(1) process $\tilde{X}_{t}(u)=\sum_{k=0}^{\infty}\alpha(u)^{k}\varepsilon_{t-k}$ with Lipschitz continuous $\alpha:[0,1]\to(-1,1)$ and $\|\varepsilon_{i}\|_{q}<\infty$, then it is easy to see that $X_{t,n}=\tilde{X}_{t}(t/n)+(\varepsilon_{t}+...+\varepsilon_{t-n})n^{-2}$ satisfies (S1) with $q=1$ and $\alpha=1$, but $\delta_{q}^{X_{\cdot,n}}(k)=\sup_{t}|\theta(t/n)^{k}+n^{-2}|$ is not absolutely summable. There are more counterexamples, see the Supplementary Material 8.

**Invariance property of the assumptions with respect to transformations: ** For some fixed $r\in\mathbb{N}$ define $Z_{t,n}:=(X_{t,n},...,X_{t-r+1,n})^{\prime}$ and $\tilde{Z}_{t}(u):=(\tilde{X}_{t}(u),...,\tilde{X}_{t-r+1}(u))^{\prime}$. We prove that $g(Z_{t,n})$ also fulfills Assumptions 2.1 and 2.3 for the following class of functions $g$.

Definition 2.4** (The class $\mathcal{L}_{r}(M,C)$).**

We say that a function $g:\mathbb{R}^{r}\to\mathbb{R}$ is in the class $\mathcal{L}_{r}(M,C)$ if $M\geq 0$ and

[TABLE]

where $|y|_{1}:=\sum_{i=1}^{r}|y_{i}|$.

Proposition 2.5** (Invariance property of the stationary approximation).**

Assume that $g\in\mathcal{L}_{r}(M,C)$ and $q>0$.

1. (i)

If Assumption 2.1(S1),(S2)* and Assumption 2.3(M1),(M2)** are fulfilled for the process $X_{t,n}$ with $\tilde{q}=q\cdot(M+1)$ and $1\geq\alpha>0$, then the same assumptions are fulfilled for the process $g(Z_{t,n})$ with $q$ and $\alpha$.*

Now let $g$ in addition be continuously differentiable where the partial derivatives $\partial_{j}g$, $j=1,...,r$ fulfill $\partial_{j}g\in\mathcal{L}_{r}(M-1,C^{\prime})$ with some $M\geq 1$.

1. (ii)

If Assumption 2.1(S3)* and 2.3(M1),(M3)** are fulfilled for the process $X_{t,n}$ with $\tilde{q}=q\cdot(M+1)$, then the same assumptions are fulfilled for the process $g(Z_{t,n})$ with $q$.*

The proof is immediate from Hoelder’s inequality and therefore omitted. Let us mention that the condition $\partial_{j}g\in\mathcal{L}_{r}(M-1,C^{\prime})$ is only needed to prove the mixing condition 2.3(M3) since $\|\sup_{u\in[0,1]}|\partial_{u}g(\tilde{Z}_{0}(u))|\|_{q}<\infty$ can be shown by using $|\partial_{j}g(z)|\leq C(1+|z|_{1}^{M})$.

With slight changes, the statements of Proposition 2.5 can be extended to Hoelder continuous functions which fulfill

[TABLE]

with some $1\geq\beta>0$.

In view of the above proposition, all theorems formulated for $X_{t,n}$ in this and the next section are, under appropriate moment conditions, also valid for transformations $g(Z_{t,n})$ of $X_{t,n}$. An important example is the covariance operator $g:\mathbb{R}^{r}\to\mathbb{R},g(x_{1},...,x_{r})=x_{1}x_{r}$ which leads to $g(Z_{t,n})=X_{t,n}X_{t-r+1,n}$ and fulfills $g\in\mathcal{L}_{r}(1,1)$.

**Local and global laws of large numbers: ** The smoothness of $X_{t,n}$ in time direction can be used to obtain laws of large numbers by only assuming the existence of the first moment of $X_{t,n}$. The key step of the proof is to split the sum over $X_{t,n}$ into sums over smaller ranges of $t$ where $X_{t,n}$ can be approximated by stationary processes. We will also provide results for localized sums. Usually, we will need the following

Assumption 2.6** (Localizing kernel).**

$K:\mathbb{R}\to\mathbb{R}$* is a bounded function, i.e. with some $|K|_{\infty}>0$ it holds that $\sup_{x\in\mathbb{R}}|K(x)|\leq|K|_{\infty}$, and of bounded variation $B_{K}$ with compact support $[-\frac{1}{2},\frac{1}{2}]$ satisfying $\int K\ \mbox{d}x=1$. Let $K_{b}(x):=\frac{1}{b}K(\frac{x}{b})$.*

The first part of the following theorem can be seen as a generalization of the ergodic theorem to non-stationary processes, while the second part provides uniform convergence rates if more than the first moment is available.

Theorem 2.7** (Law of large numbers).**

Let $q=1$ in (i),(ii) and $q>1$ in (iii). Suppose that Assumption 2.1(S1) holds with some $1\geq\alpha>0$ and that Assumption 2.6 holds. Then we obtain for the process $X_{t,n}$ (or alternatively for the process $g(Z_{t,n})$ if Assumption 2.1(S1) is fulfilled for the process $X_{t,n}$ with $\tilde{q}=q\cdot(M+1)$ instead of $q$) the following results:

1. (i)

[TABLE] 2. (ii)

For each $u\in(0,1)$

[TABLE]

as $n\to\infty$, $nb\to\infty$ and $b=b_{n}\to 0$. 3. (iii)

Additionally, suppose that Assumption 2.3(M1)* holds with $q,\alpha$. Then*

[TABLE]

If $q>2$, then there exist constants $C_{1},C_{2}$ not depending on $n,b$ such that for all $x>0$:

[TABLE]

with positive constants $C_{1},C_{2}$ not depending on $n,b$ and $G_{\gamma}(y):=\sum_{j=1}^{\infty}e^{-j^{\gamma}y^{2}}$ a Gaussian-like tail function. $G_{\gamma}(y)$ can be replaced by $\exp(-cy^{2})$ for some $c>0$.

Remark 2.8**.**

1. (i)

The additional $O(n^{-\alpha}b^{-1})$ or $O((n^{-\alpha}b^{-1})^{\tilde{q}})$ terms in (8) and (9), respectively, can be omitted under Assumption 2.3(M2)**. In this case, one has to replace $\Delta_{0,\tilde{q}}^{\tilde{X}}$ by $\Delta_{0,\tilde{q}}^{X}$. 2. (ii)

For $q>1$, $b=o(n^{1-\frac{1}{q}})$ and $n=o(b^{1/\alpha})$, the results of Theorem 2.7(ii) and Proposition 3.3 (which is about bias expansion) can be used to obtain uniform convergence of the mean estimator \hat{\mu}_{b}(u):=\frac{1}{nb}\sum_{t=1}^{n}K\big{(}\frac{t/n-u}{b}\big{)}X_{t,n} towards $\mu(u):=\mathbb{E}\tilde{X}_{0}(u)$ in the sense that

[TABLE]

A central limit theorem: We provide local and global central limit theorems under minimal moment conditions which are useful in particular to find asymptotic distributions of (nonparametric) estimators of locally stationary processes, see Section 5. For the proofs we need the dependence condition from Assumption 2.3(M1). Assumption 2.1(S2), namely $\|\sup_{u\in[0,1]}|\tilde{X}_{t}(u)|\|_{q}<\infty$, is crucial to show a Lindeberg-type condition under minimal moment conditions.

Theorem 2.9** (Central limit theorem - global version).**

Let Assumption 2.6 hold. Suppose that Assumption 2.1(S1), (S2) and 2.3(M1) hold with some $q\geq 2$ and $\alpha>\frac{1}{2}$. Define S_{n}:=\sum_{t=1}^{n}\big{(}X_{t,n}-\mathbb{E}X_{t,n}\big{)} (if $X_{t,n}$ is replaced by $g(Z_{t,n})$ in the assertions below the same assumptions must be fulfilled with $\tilde{q}=q\cdot(M+1)$ instead of $q$). Then we have the following invariance principle:

[TABLE]

where $B(v)$ is a standard-Brownian motion and the long-run variance $\sigma^{2}(v)$ is given by

[TABLE]

The condition $\alpha>\frac{1}{2}$ can be omitted under Assumption 2.3(M2).

Finally, we present a simple localized version of the central limit theorem for general locally stationary processes.

Theorem 2.10** (Central limit theorem - local version).**

Let Assumption 2.6 hold. Suppose that Assumption 2.1(S1), (S2) and 2.3(M1) hold with some $q\geq 2$ and $1\geq\alpha>0$ (if $X_{t,n}$ is replaced by $g(Z_{t,n})$ in the assertion below the same assumptions must be fulfilled with $\tilde{q}=q\cdot(M+1)$ instead of $q$). Then for all $u\in(0,1)$, provided that $\sqrt{nb}\cdot n^{-\alpha}\to 0$, $b\to 0$ and $nb\to\infty$:

[TABLE]

*with $\sigma^{2}(u)$ defined in (10). *

3 Differential calculus for nonstationary processes

In this section we prove almost sure and uniform $L^{q}$ Taylor expansions of $X_{t,n}$ and invent a kind of differential calculus for locally stationary processes. Below we will use this stronger kind of approximation for proving deterministic and stochastic bias expansions. While deterministic bias expansions are used to bound the expectation of expressions which include $X_{t,n}$, stochastic bias expansions can be used to replace the whole localized sum by a localized sum of the stationary process $\tilde{X}_{t}(u)$ which is much easier to analyze with known tools. We also prove results on the spectrum and the empirical distribution function.

3.1 Taylor expansions and expansions of localized sums

Proposition 3.1** (Taylor expansion).**

Suppose that Assumption 2.6 holds. Suppose that Assumption 2.1 holds for some $q>0$. Then we have for all $u\in[0,1]$ and $t=1,...,n$:

[TABLE]

If $|\frac{t}{n}-u|=o(1)$, then it holds that $R_{t,n}=o_{a.s.}(|\frac{t}{n}-u|)$. Furthermore,

[TABLE]

holds and $|R_{n}|\leq|K|_{\infty}\cdot b\cdot\sup_{|v-u|\leq b}|\partial_{u}X_{t}(v)-\partial_{u}X_{t}(u)|=o_{a.s.}(b)$.

If one is interested in the approximation of moments of localized sums (as it may be the case in bias expansions in nonparametric frameworks), the following theorem is appropriate. It also closes the gap between the locally stationary process $X_{t,n}$ and its approximation $\tilde{X}_{t}(t/n)$.

Corollary 3.2** (Almost sure and $L^{q}$ expansion of localized sums).**

Suppose that Assumption 2.6 holds. Suppose that Assumption 2.1 holds for some $q\geq 1$. Then for each fixed $u\in(0,1)$,

[TABLE]

and the expansion (12) is valid with $\|R_{n}\|_{q}=o(b)$.

It is also possible to use expansions similar to above if the sum is not localized by a kernel. An example can be found in the proof of Theorem 2.7.

3.2 Bias Expansions

In nonparametric statistics, bias expansions play an important role to control the mean squared error (MSE) of estimators. Here we give an approach to estimate the deterministic bias term involving locally stationary processes. In recent years (for instance due to model selection via contrast minimization) a more careful analysis of the stochastic part in the calculation of the MSE became important. We make a contribution to this topic via a stochastic bias expansion which allows us to remove the bias from a localized sum \frac{1}{n}\sum_{t=1}^{n}K_{b}\big{(}\frac{t}{n}-u\big{)}X_{t,n} such that only a localized sum over a stationary process \frac{1}{n}\sum_{t=1}^{n}K_{b}\big{(}\frac{t}{n}-u\big{)}\tilde{X}_{t}(u) remains for which much more theoretical work was done. To obtain bias expansions of smaller order than $O(b)$ we have to assume differentiability of the upcoming expectations which is ensured by assuming Assumption 2.1(S3). To emphasize some differences that occur when Assumption 2.1(S3) is changed to differentiability in $L^{q}$, we state the deterministic bias expansion for both settings and comment in Remark 3.5.

Proposition 3.3** (Deterministic bias expansion).**

Suppose that Assumption 2.6 holds. Let $q\geq 1$. Suppose that Assumption 2.1(S1) is fulfilled with some $1\geq\alpha>0$, then we have uniformly in $u\in[0,1]$:

[TABLE]

and

[TABLE]

Now assume additionally that $K$ is symmetric.

1. (a)

If Assumption 2.1(S3)* holds, then (13) is valid with $\alpha=1$ and we have uniformly in $u\in[\frac{b}{2},1-\frac{b}{2}]$*

[TABLE] 2. (b)

Suppose that $[0,1]\to L^{q},u\mapsto\tilde{X}_{0}(u)$ is Fréchet differentiable with derivative $\tilde{D}_{t}(u)$, i.e. for all $u\in[0,1]$,

[TABLE]

Then the statement of (a) holds.

The proof of (13) and (14) follows from the Hoelder inequality and the fact that $K$ has bounded variation and thus is bounded. To prove (15), note that

[TABLE]

As long as $|\frac{t}{n}-u|\leq b$, we have

[TABLE]

since $u\mapsto\partial_{u}\tilde{X}_{t}(u)$ is continuous and $\|\sup_{u}|\partial_{u}\tilde{X}_{t}(u)|\|_{1}<\infty$. Finally, because $K$ has bounded variation and is symmetric,

[TABLE]

The proof of (b) follows by using

[TABLE]

Remark 3.4**.**

Note that in the situation of Proposition 3.3, derivative processes were used to get $o(b)$ instead of $O(b)$ in (14). Even smaller rates can be obtained by using higher order derivative processes together with higher order kernels. If we assume that $u\mapsto\tilde{X}_{t}(u)$ has a twice continuously differentiable modification and $K$ is symmetric, we obtain a bias decomposition whose structure is well-known:

[TABLE]

Remark 3.5** (Almost sure v.s. $L^{q}$ differentiability).**

Let us briefly comment on the different conditions in Proposition 3.3(a), (b). In (a) we ask for a.s. differentiability of $u\mapsto\tilde{X}_{0}(u)$ while (b) asks for differentiability in $L^{q}$ which is weaker. If we want to apply Proposition 3.3 to $g(Z_{t,n})$ with some function $g\in\mathcal{L}_{r}(M,C)$ which is continuously differentiable, then there occur some differences due to the different natures of the conditions:

• •

If Assumption 2.1(S3)* is fulfilled for $q^{\prime}=q(M+1)$, then Assumption 2.1(S3)** is fulfilled for the process $g(Z_{t,n})$ with $q$ (cf. Proposition 2.5(ii) and the comment afterwards) and we obtain (15) by Proposition 3.3(a).*

• •

If (16) holds, we have to assume additionally that all the derivatives $\partial_{j}g$ ($j=1,...,r$) are Hoelder continuous with polynomially growing Hoelder constant, i.e. with some $\gamma>0$,

[TABLE]

to obtain (16) for $\tilde{X}_{t}^{\circ}(u)=g(\tilde{Z}_{t}(u))$ with derivative $D_{t}^{\circ}(u)=\partial_{z}g(\tilde{Z}_{t}(u))\cdot D_{t}(u)$.

Note that we have to ask $g$ to be slightly more smooth when using differentiability in $L^{q}$.

We now prove the stochastic bias expansion. It turns out that we have to bound moments of sums of the upcoming derivative processes $\partial_{u}\tilde{X}_{t}(u)$ which means that we have to pose dependence conditions on $\partial_{u}\tilde{X}_{t}(u)$. This is done via Assumption 2.3(M3). Using the projection operator $P_{j}\cdot:=\mathbb{E}[\cdot|\mathcal{F}_{j}]-\mathbb{E}[\cdot|\mathcal{F}_{j-1}]$, we can bound moments of sums of $\partial_{u}\tilde{X}_{t}(u)$ by moments of martingales which can then be bounded with results from Rio, E. (2009). It can be shown (similar to Wu, W.B. (2005), Theorem 1(i) and (ii)) that for some shift process $W_{t}=H_{t}(\mathcal{F}_{t})$ with measurable $H_{t}$ it holds for $q\geq 1$:

[TABLE]

Proposition 3.6** (Stochastic bias expansion).**

Suppose that Assumption 2.1 and 2.3(M3) are fulfilled for some $q\geq 2$. Assume that $K$ is symmetric. Then we have

[TABLE]

Proof of Proposition 3.6:.

To prove (18), we can show similarly as in (13) that

[TABLE]

To deal with \frac{1}{n}\big{\|}\sum_{t=1}^{n}K_{b}\big{(}\frac{t}{n}-u\big{)}(\tilde{X}_{t}(t/n)-\tilde{X}_{t}(u))\big{\|}_{q}, we use the expansion (12) together with the result $\|R_{n}\|_{q}=o(b)$ from Corollary 3.2. It therefore remains to analyze

[TABLE]

While the second term is $O(n^{-1})$ by stationarity, the first term is bounded by

[TABLE]

by Theorem 2.1 in Rio, E. (2009) and (17). ∎

The main advantage of a stochastic bias expansion is that we can reduce a sum over locally stationary processes to a sum over stationary processes by keeping the terms stochastic. This allows for instance to apply large deviation results for stochastic processes which usually have a simpler and more closed form.

Remark 3.7** (An application of the stochastic bias expansion: Deviation inequalities).**

Suppose that the assumptions of Proposition 3.6 are fulfilled with $q=2$. Assume that $\mathbb{E}\tilde{X}_{t}(u)=0$. Then we have for $\gamma>0$,

[TABLE]

By Proposition 3.6, the second term on the right hand side is $o(b)+O(n^{-1})$, i.e. has bias order. For the first term, one can use deviation results for stationary processes.

3.3 Differentiability of functionals

The existence of derivative processes allows an expansion of the corresponding mean $\mathbb{E}g(Z_{t,n})$ into the mean of the corresponding stationary version $\mathbb{E}g(\tilde{Z}_{t}(u))$. This can be applied to various functionals such as expectations, covariances, the Wigner-Ville spectrum and the distribution function. The following result is an immediate Corollary from Lemma 2.5 applied to some $g\in\mathcal{L}_{r}(M,C)$. Recall $Z_{t,n}=(X_{t,n},...,X_{t-r+1,n})^{\prime}$ and $\tilde{Z}_{t}(u)=(\tilde{X}_{t}(u),...,\tilde{X}_{t-r+1}(u))^{\prime}$.

Proposition 3.8**.**

Assume that $g\in\mathcal{L}_{r}(M,C)$. Suppose that Assumption 2.1(S1) is fulfilled for some $1\geq\alpha>0$ and $q=M+1$. Then we have uniformly for $t=1,...,n$:

[TABLE]

If additionally Assumption 2.1(S3) is fulfilled and $g$ is continuously differentiable with partial derivatives $\partial_{j}g\in\mathcal{L}_{r}(M-1,C^{\prime})$, $j=1,...,r$, then $u\mapsto\mathbb{E}g(\tilde{Z}_{t}(u))$ is continuously differentiable with derivative

[TABLE]

The result of Proposition 3.8 enables us to get expansions of the mean, the covariance and the distribution function of $X_{t,n}$. Suppose in the following that Assumption 2.1 holds for some $q\geq M+1$.

Corollary 3.9** (Mean expansion, $M=0$).**

Choosing $g:\mathbb{R}\to\mathbb{R},g(y)=y$ yields

[TABLE]

where $\mu(u):=\mathbb{E}\tilde{X}_{0}(u)$ is continuously differentiable with derivative $\partial_{u}\mu(u)=\mathbb{E}\partial_{u}\tilde{X}_{0}(u)$.

Corollary 3.10** (Covariance expansion, $M=1$).**

Fix $r>0$. Define the covariances $\gamma(u,r):=\mbox{Cov}(\tilde{X}_{t}(u),\tilde{X}_{t-r}(u))$. Choosing $g:\mathbb{R}^{r+1}\to\mathbb{R},g(y)=y_{1}y_{r+1}$, we obtain uniformly for $t=1,...,n$:

[TABLE]

and $\gamma(u,r)$ is continuously differentiable with derivative

[TABLE]

Similar expansions can be derived for higher-order cumulants and also for the Wigner-Ville spectrum (cf. Martin, W. and Flandrin, P. (1985)).

As a last application of Proposition 3.8, we present an expansion of the distribution function of $X_{t,n}$ which may also be used to approximate quantiles of locally stationary processes.

Example 3.11** (Expansion of the distribution function).**

Suppose that Assumption 2.1 holds with $q=1$. Assume that the i.i.d. random variables $\varepsilon_{t}$, $t\in\mathbb{Z}$ have a Lipschitz continuous and continuously differentiable distribution function $F_{\varepsilon}$ with Lipschitz constant $L_{\varepsilon}$ and derivative $f_{\varepsilon}$. Let the processes $X_{t,n}$ and $\tilde{X}_{t}(u)$ obey the recursion equations (3) and (4).

Assume that $(\varepsilon,y,u)\mapsto G_{\varepsilon}(y,u)$ is continuously differentiable and that the derivative $\partial_{\varepsilon}G_{\varepsilon}(y,u)\geq\delta_{G}>0$ is uniformly bounded from below by some positive constant $\delta_{G}>0$. By the inverse function theorem we know that there exists a continuously differentiable inverse $x\mapsto H(x,y,u)$ of $\varepsilon\mapsto G_{\varepsilon}(y,u)$. Finally, assume that for all $x\in\mathbb{R}$, the expressions

[TABLE]

are finite.

Put $Y_{t-1,n}=(X_{t-1,n},...,X_{t-p,n})^{\prime}$, $\tilde{Y}_{t-1}(u)=(\tilde{X}_{t-1}(u),...,\tilde{X}_{t-p}(u))^{\prime}$. In this situation it holds that the distribution function of $X_{t,n}$,

[TABLE]

can be approximated by the distribution function $F_{\tilde{X}_{t}(u)}(x):=\mathbb{P}(\tilde{X}_{t}(u)\leq x)$ by

[TABLE]

Furthermore $u\mapsto F_{\tilde{X}_{t}(u)}(x)$ is differentiable with derivative

[TABLE]

4 Nonlinear locally stationary processes

In this section we show in a sequence of theorems that the Markov processes given by (3) and (4) fulfill Assumption 2.1 and the mixing conditions of Assumption 2.3. Furthermore we prove that the derivative process can be obtained as the solution of a functional equation. The existence of these processes and their properties have previously been derived for tvAR models (cf. Dahlhaus, R. (2012)), tvARCH models (cf. Dahlhaus, R., and Subba Rao, S. (2006)) and random coefficient models (cf. Subba Rao, S. (2006)). The situation in the present case is however different since the process (3) is only defined by a recursion and the explicit solution is usually not available for the calculations. To prove the results, we state the following elementary assumptions on the recursion function $G_{\varepsilon}(y,u)$. Let $\partial_{1}G$, $\partial_{2}G$ denote the derivatives of $G$ w.r.t. $y$ and $u$, respectively.

Assumption 4.1**.**

In the model (3), (4) we assume with $Y_{t-1,n}=(X_{t-1,n},...,X_{t-p,n})^{\prime}$ and $\tilde{Y}_{t-1}(u)=(\tilde{X}_{t-1}(u),...,\tilde{X}_{t-p}(u))^{\prime}$ that there exists $q>0$, $\chi=(\chi_{1},...,\chi_{p})\in\mathbb{R}^{p}_{\geq 0}$ with $|\chi|_{1}=\sum_{i=1}^{p}\chi_{i}<1$ and $y_{0}\in\mathbb{R}^{p}$ such that with $q^{\prime}:=\min\{q,1\}$:

1. (L1)

$\sup_{u\in[0,1]}\|G_{\varepsilon_{0}}(y_{0},u)\|_{q}<\infty$, and (with |z|_{\chi,q^{\prime}}:=\big{(}\sum_{i=1}^{p}|z_{i}|^{q^{\prime}}\cdot\chi_{i}\big{)}^{1/q^{\prime}} the weighted $q^{\prime}$-norm)

[TABLE] 2. (L2)

$(y,u)\mapsto G_{\varepsilon}(y,u)$* is continuous for all $\varepsilon$, $\|\sup_{u\in[0,1]}|G_{\varepsilon_{0}}(y_{0},u)|\ \|_{q}<\infty$, and*

[TABLE] 3. (L3)

$(y,u)\mapsto G_{\varepsilon}(y,u)$* is continuously differentiable for all $\varepsilon$, $\|\sup_{u\in[0,1]}|\partial_{2}G_{\varepsilon_{0}}(y_{0},u)|\ \|_{q}<\infty$, and*

[TABLE]

Furthermore, assume that either (a) (23) holds for $q/2$ instead of $q$ or (b) $y\mapsto\partial_{1}G_{\varepsilon}(y,u)$ is constant for all $\varepsilon,u$. 4. (L4)

For some $0<\alpha\leq 1$, it holds that

[TABLE]

Let us briefly discuss the conditions in Assumption 4.1.

Remark 4.2**.**

• (i)

Note that (L1)-(L3) impose increasingly strong smoothness assumptions on the recursion function $G_{\varepsilon}(y,u)$. While (L1)-(L3) are directly verifiable, (L4) includes conditions on the stationary approximation $\tilde{X}_{t}(u)$. Note that the upcoming theorems also state properties of $\tilde{X}_{t}(u)$. Their results can be used to verify (L4).

• (ii)

The condition (L2) means that the mapping $y\mapsto G_{\varepsilon}(y,u)$ can be viewed as a contraction in the space of continuous functions $C[0,1]$ which in turn implies the a.s. continuity of the limit. (L3) is necessary to ensure that $y\mapsto G_{\varepsilon}(y,u)$ is a contraction in $C^{1}[0,1]$.

• (iii)

Condition (L3)(a) or (b) is necessary due to the product in (28). Note that by Hoelder’s inequality, (L3)(a) follows from (23) if $q\geq 2$. The inequality $|z|_{q}\leq|z|_{q^{\prime}}$ for $0<q^{\prime}\leq q$, $z\in\mathbb{R}^{p}$ implies that (L3)(a) is fulfilled if (23) holds with $\sum_{i=1}^{p}\chi_{i}^{1/2}<1$.

• (iv)

For $p>1$, the conditions stated in Assumption 4.1 may lead to non-optimal restrictions on $G$ which is due to the general formulation. One way to circumvent this is by posing conditions on the $m$-th iteration of $G$ instead of $G$ itself. Some models like tvAR($p$) or tvARCH($p$) also allow a reformulation to a $p$-dimensional recursion with only one lag. Since our aim is to cover a wide range of models with simple conditions, we will not discuss these approaches in detail.

Remark 4.3** (Almost sure calculus v.s. $L^{q}$ calculus).**

As mentioned in the beginning of this paper, many of the statistical applications in Section 3 can be proved by only assuming differentiability of $u\mapsto\tilde{X}_{t}(u)$ in $L^{q}$, an example was given in Proposition 3.3(b). As pointed out by a referee, to obtain $L^{q}$ differentiability, Assumptions 4.1(L2), (L3) can be weakened. Technically, proving differentiability of $u\mapsto\tilde{X}_{t}(u)$ then corresponds to analyzing smoothness properties of a fixed point of the iteration (4). As analytic results in Sedro, J. (2017) suggest, the main difference is a decrease in smoothness assumptions that have to be posed on $G$, namely $G$ is no longer needed to be continuously differentiable but only differentiable a.e. and the suprema in (23) and (24) can be taken outside. This theory would also include $tvTAR$ processes (cf. Example 2.2).

There are some drawbacks when using only $L^{q}$ calculus and no a.s. statements. As already mentioned in Remark 3.5, one has to pose slightly more smoothness conditions on $g$ if one wants to apply the theory to processes $g(Z_{t,n})$. Moreover, it seems that a Lindeberg-type condition which is used in the proof of the global CLT Theorem 2.9 can only be shown under Assumption 2.1(S2) when only second moments are available. To ensure 2.1(S2), we have to ask for 4.1(L2) in view of Theorem 4.6.

Existence and uniqueness of $X_{t,n}$ and $\tilde{X}_{t}(u)$. We now establish existence and uniqueness under mild contraction conditions.

Proposition 4.4**.**

(i) Existence of a stationary approximation: Suppose that Assumption 4.1(L1) holds. Then for all $u\in[0,1]$, the recursion (4) has an a.s. unique $\mathcal{F}_{t}$-measurable, stationary and ergodic solution $\tilde{X}_{t}(u)=H(u,\mathcal{F}_{t})$ and we have with some $C>0$ and $0<\rho<1$:

[TABLE]

(ii) Existence of the nonstationary process: Under the above conditions, there exists an a.s. unique $\mathcal{F}_{t}$-measurable solution of (3) with $X_{t,n}=H_{t,n}(\mathcal{F}_{t})$, where $H_{t,n}$ are measurable functions. Furthermore, $\sup_{n\in\mathbb{N}}\sup_{t=1,...,n}\|X_{t,n}\|_{q}<\infty$ and with some $C>0$ and $0<\rho<1$:

[TABLE]

The proof of (i) for fixed $u\in[0,1]$ is similar to the proof in Shao, X., and Wu, W.B. (2007), Theorem 5.1. Since we state the results uniformly in $u\in[0,1]$, we will give the proof in the appendix for completeness. Since the definition of $X_{t,n}$ and $\tilde{X}_{t}(0)$ coincide for $t\leq 0$, existence and uniqueness of $X_{t,n}$ follow from the existence and uniqueness of $\tilde{X}_{t}(0)$. Therefore, the existence statement in (ii) is an immediate corollary of (i).

A uniform $L^{q}$ approximation: We now prove that $X_{t,n}$ can be approximated by the stationary process $\tilde{X}_{t}(u)$ uniformly in a $L^{q}$-sense.

Lemma 4.5**.**

Suppose that Assumption 4.1(L1),(L4) hold. Then

[TABLE]

Furthermore, we have:

[TABLE]

Note that the approximation error in (27) cannot be avoided - cf. Dahlhaus, R. (2012), (49), for the tvAR(1) case (with a different error due to different assumptions).

Existence of continuous modifications and derivative processes: Proposition 4.4 gives the almost sure uniqueness of $\tilde{X}_{t}(u)$ for each $u\in[0,1]$, but not continuity of $u\mapsto\tilde{X}_{t}(u)$ since this involves uncountably many points $u\in[0,1]$. In order to guarantee the existence of a continuous or even differentiable modification $\hat{X}_{t}(u)$ of $\tilde{X}_{t}(u)$ we have to impose stronger conditions on the recursion function $G$ in (3) ($\hat{X}_{t}(u)$ is a modification of $\tilde{X}_{t}(u)$ if for all $u\in[0,1]$, $\hat{X}_{t}(u)=\tilde{X}_{t}(u)$ a.s.). A natural way would be to apply extensions of the Kolmogorov-Chentzov theorem, but they usually contain tradeoffs in their conditions between moment assumptions and smoothness of the process which usually leads to either strong moment or smoothness assumptions which may not be useful in practice. Furthermore it does not use the specific structure of the process which is known and we could not give a bound for moments of $\sup_{u\in[0,1]}|\hat{X}_{t}(u)|$. We therefore use a different approach.

Theorem 4.6** (Existence of a continuous modification).**

Suppose that Assumption 4.1(L2) holds. Then for each $t\in\mathbb{Z}$, there exists a continuous modification $(\hat{X}_{t}(u))_{u\in[0,1]}$ of $(\tilde{X}_{t}(u))_{u\in[0,1]}$ from Proposition 4.4 with $\sup_{u\in[0,1]}|\hat{X}_{t}(u)|\in L^{q}$.

Remark 4.7**.**

In the case $G_{\varepsilon}(y,u)=\tilde{G}_{\varepsilon}(y,\theta_{0}(u))$ with some parameter curve $\theta_{0}:[0,1]\to\Theta$ (cf. Section 5), the supremum taken over $u\in[0,1]$ in (23) restricts the parameter space $\Theta$. If additionally 4.1(L3) is fulfilled, Theorem 4.6 also holds under the weaker condition \sup_{u\in[0,1]}\Big{\|}\sup_{y\not=y^{\prime}}\frac{|G_{\varepsilon_{0}}(y,u)-G_{\varepsilon_{0}}(y^{\prime},u)|}{|y-y^{\prime}|_{\chi,q^{\prime}}}\Big{\|}_{q}\leq 1 which leads to larger admissible parameter spaces $\Theta$. For details see Proposition 7.3 in the appendix.

In the following we assume that $(y,u)\mapsto G_{\varepsilon}(y,u)$ is differentiable in both components. For the moment, assume that there exists a modification $(\hat{X}_{t}(u))_{u\in[0,1]}$ of the process $(\tilde{X}_{t}(u))_{u\in[0,1]}$ with differentiable paths and denote the derivative by $\partial_{u}\hat{X}_{t}(u)$. Then the following recursion equation for $D_{t}(u)=\partial_{u}\hat{X}_{t}(u)$, obtained by differentiating (4) should hold a.s.:

[TABLE]

This is shown in the next theorem. The first part is devoted to the existence of a solution $D_{t}(u)$ of the recursion (28) given the existence of the process $\tilde{X}_{t}(u)$ from Theorem 4.4; in the second part we prove that $\tilde{X}_{t}(u)$ has a differentiable modification with respect to $u$ and that the derivative coincides with $D_{t}(u)$. Both $\tilde{X}_{t}(u)$ and $D_{t}(u)$ are uniquely determined by (4) and (28).

Theorem 4.8** (Existence of derivative processes).**

Suppose that Assumptions 4.1(L2), (L3) hold. Then the following statements are true.

1. (i)

Existence of the first derivative process: For all $u\in[0,1]$, the recursion (28) has a unique stationary and ergodic solution $D_{t}(u)=\tilde{H}(u,\mathcal{F}_{t})$ with some measurable $H$ and it holds that

[TABLE]

with some $C>0$, $0<\rho<1$. 2. (ii)

Differentiability:

• (a)

There exists a continuously differentiable modification $(\hat{X}_{t}(u))_{u\in[0,1]}$ of the process $(\tilde{X}_{t}(u))_{u\in[0,1]}$ from Proposition 4.4 such that for all $u\in[0,1]$ it holds that $\partial_{u}\hat{X}_{t}(u)=D_{t}(u)$ a.s.

• (b)

$\sup_{u\in[0,1]}|\partial_{u}\hat{X}_{t}(u)|\in L^{q}$.

Finally, let us summarize the results from this section in the following Corollary.

Corollary 4.9**.**

Let Assumption 4.1 be fulfilled. Then modifications of the a.s. unique solutions of (3) and (4) fulfill Assumption 2.1 and 2.3.

For some models it is possible to obtain explicit expressions for the corresponding derivative processes.

Example 4.10** (Explicit representations for derivative processes).**

1. (i)

The tvAR($p$) process X_{t,n}=\sum_{j=1}^{p}a_{j}\big{(}\frac{t}{n}\big{)}X_{t-j,n}+\varepsilon_{t} has the corresponding stationary approximation $\tilde{X}_{t}(u)=\sum_{j=1}^{p}a_{j}(u)\tilde{X}_{t-j}(u)+\varepsilon_{t}$ which has an explicit representation $\tilde{X}_{t}(u)=\sum_{j=0}^{\infty}\psi_{j}(u)\cdot\varepsilon_{t-j}$ with differentiable $\psi_{j}:[0,1]\to\mathbb{R}$ ($j=0,1,2,...)$. It is easy to see that $\partial_{u}\tilde{X}_{t}(u)=\sum_{j=0}^{\infty}\partial_{u}\psi_{j}(u)\cdot\varepsilon_{t-j}$ is the a.s. uniquely determined derivative process. 2. (ii)

Similarly to (i), it is easy to see that general linear processes $\tilde{X}_{t}(u)=\sum_{j=0}^{\infty}\psi_{j}(u)\cdot\varepsilon_{t-j}$ with differentiable $\psi_{j}:[0,1]\to\mathbb{R}$ ($j=0,1,2,...$) have derivative process $\partial_{u}\tilde{X}_{t}(u)=\sum_{j=1}^{\infty}\partial_{u}\psi_{j}(u)\cdot\varepsilon_{t-j}$ under appropriate summability conditions. 3. (iii)

For tvARCH($p$) processes, explicit expressions for the derivative processes were obtained in Dahlhaus, R., and Subba Rao, S. (2006)**.

In the following we will write $\tilde{X}_{t}(u)$ even if we mean the differentiable modification to keep notation simple. Since all our results only involve countably many observations, this will not cause any problems.

Higher order derivative processes: Under additional assumptions, one can show uniform $L^{q}$ Hoelder properties of the first derivative process:

Proposition 4.11** (Hoelder property of the first derivative process).**

Suppose that Assumption 4.1(L2),(L3) hold. Additionally assume that for some $1\geq\alpha_{2}>0$ and $i=1,2$ it holds component-wise:

[TABLE]

Then

[TABLE]

with some constant $C>0$.

If $\tilde{X}_{t}(u)$ has a twice continuously differentiable modification and $(y,u)\mapsto G_{\varepsilon}(y,u)$ is twice continuously differentiable, then the following recursion equation for $\partial_{u}^{2}\tilde{X}_{t}(u)$ should hold:

[TABLE]

Using the same techniques as in Theorem 4.8, one can find similar conditions as in Assumption 4.1 such that a second (or even higher) order derivative process $\partial_{u}^{2}\tilde{X}_{t}(u)$ exists. Let us point out an interesting anomaly in the case of second order derivatives that is also existent for higher order derivatives: Due to the additional products in (30) it turns out that, in general, one has to assume $2q$-th moments of $\tilde{X}_{t}(u)$ to guarantee the existence of the $q$-th moment of $\partial_{u}^{2}\tilde{X}_{t}(u)$. The formalization of this is beyond the scope of this paper, but in Proposition 4.11 one already can see the imbalance of moments in the assumption and the obtained result.

**A simulation study: ** To quantify the quality of the approximations given in Lemma 4.5 and Proposition 3.1, we consider the tvARCH(1) model

[TABLE]

with $a_{0}:=0.2$, $a_{1}(u)=0.95u^{2}$ and $\varepsilon_{0}\sim N(0,1)$. Note that if $t/n$ tends to 1, the values of $X_{t,n}$ are more dependent to each other than for smaller values of $t/n$. We generated realizations of $X_{t,n}$, $\tilde{X}_{t}(\frac{t}{n})$ with $n=500$ (see Figure 1(a),(b) for a realization of $X_{t,n}$ and $X_{t,n}-\tilde{X}_{t}(\frac{t}{n})$). In Figure 1(c) we have the plotted empirical 5%- and 95%-quantile curves of the difference $X_{t,n}-\tilde{X}_{t}(\frac{t}{n})$ for $N=1000$ replications. It can be seen that with stronger dependence, the quality of the approximation $X_{t,n}\approx\tilde{X}_{t}(\frac{t}{n})$ gets worse as it is suggested by the bound in Lemma 4.5.

Secondly we consider the approximation quality of $\tilde{X}_{t}(t/n)$ by $\tilde{X}_{t}(u)$ and $\tilde{X}_{t}(u)+(\frac{t}{n}-u)\partial_{u}\tilde{X}_{t}(u)$, respectively. Since these approximations are only working locally (for $|t/n-u|\ll 1$), we compare them by dividing the whole time line $t=1,...,n$ into subsets $(u_{i}-b,u_{i}+b]$, where $b=25$ and $u_{i}=(2i-1)b$ for $i=1,...,10$. In Figure 1(d) empirical 5%- and 95%-quantile curves obtained from $N=1000$ replications for the differences $\tilde{X}_{t}(\frac{t}{n})-\tilde{X}_{t}(u_{i})$ and $\tilde{X}_{t}(\frac{t}{n})-\tilde{X}_{t}(u_{i})-(\frac{t}{n}-u_{i})\partial_{u}\tilde{X}_{t}(u_{i})$ (where $t\in(u_{i}-b,u_{i}+b]$) are depicted, respectively. We emphasize that the improvement of the (pointwise) approximation $\tilde{X}_{t}(\frac{t}{n})$ by taking into account the derivative process is remarkable. However, both approximations again get worse if the dependence of $X_{t,n}$ to earlier values increases.

5 Application to Maximum Likelihood estimation

In this section we investigate the asymptotic properties of maximum likelihood estimates for parameter curves of locally stationary models which can be written in the form (3). The results are in particular derived by using the asymptotic results and the differential calculus of Section 2 and 3. More precisely we investigate the recursively defined model

[TABLE]

where now the function $G_{\varepsilon}(y,u)$ from (3) has been replaced by $\tilde{G}_{\varepsilon}(y,\theta_{0}(u))$ with the unknown parameter curve $\theta_{0}:[0,1]\to\Theta\subset\mathbb{R}^{d}$ which is to be estimated. Our goal is to obtain estimators for $\theta_{0}(\cdot)$ based on $X_{t,n}$, $t=1,...,n$ with a quasi maximum likelihood approach.

Suppose for the moment that $\varepsilon\mapsto G_{\varepsilon}(y,\theta)$ is continuously differentiable for all $\varepsilon,y,u$ and that the derivative $\partial_{\varepsilon}\tilde{G}_{\varepsilon}(y,\theta)\geq\delta_{G}>0$ is bounded uniformly from below with some constant $\delta_{G}>0$. This ensures that the new innovation $\varepsilon_{t}$ has an impact on the value of $X_{t,n}$ which is not too small. Under these conditions, there exists a continuously differentiable inverse $x\mapsto H(x,y,\theta)$ of $\varepsilon\mapsto G_{\varepsilon}(y,\theta)$ (see also Example 3.11).

Suppose that $\varepsilon_{0}$ has a continuous density $f_{\varepsilon}$. The negative conditional log likelihood of $X_{t,n}=x$ given $(X_{t-1,n},...,X_{t-p,n})=y$ and $\theta_{0}(\cdot)\equiv\theta$ is then

[TABLE]

In the following derivations, we do not make use of the specific structure of $\ell$. This means especially that we allow for model misspecifications due to a false density $f_{\varepsilon}$. Many authors prefer the case of a Gaussian density $f_{\varepsilon}(x)=(2\pi)^{-1/2}\exp(-x^{2}/2)$ because then a minimizer $\theta$ of $\ell$ can be interpreted as a minimum (quadratic) distance estimator (see Dahlhaus, R., and Giraitis, L. (1998) in the tvAR case, Dahlhaus, R., and Subba Rao, S. (2006) in the tvARCH case).

Based on this we define $\ell_{t,n}(\theta):=\ell(X_{t,n},Y_{t-1,n},\theta)$. Let $b\in(0,1)$ be a bandwidth $K$ a kernel function as considered in Assumption 2.6. We define the local negative log conditional likelihood

[TABLE]

For $u\in[0,1]$, the estimator of $\theta_{0}(u)$ is defined via

[TABLE]

**Asymptotic results: ** We will now discuss conditions such that $\hat{\theta}_{b}(\cdot)$ is consistent and asymptotically normal. A convenient way to formulate these results is to make a structural assumption on $\ell$: We suppose that $\ell$ is Lipschitz continuous in its components with at most polynomially increasing Lipschitz constant. To make this more precise, we introduce the class $\tilde{\mathcal{L}}_{p+1}(M,C)$ of functions using the definition of $\mathcal{L}_{r}(M,C)$ from Definition 2.4.

Definition 5.1** (The class $\tilde{\mathcal{L}}_{p+1}(M,C)$).**

We say that a function $g:\mathbb{R}^{p+1}\times\Theta\to\mathbb{R}$ is in the class $\tilde{\mathcal{L}}_{p+1}(M,C)$ with $C=(C_{z},C_{\theta})$ and constants $C_{z},C_{\theta}\geq 0$ and $M\geq 0$ if for all $z\in\mathbb{R}^{p+1},\theta\in\Theta$ it holds that $g(\cdot,\theta)\in\mathcal{L}_{p+1}(M,C_{z})$ and g(z,\cdot)\in\mathcal{L}_{d}\big{(}0,C_{\theta}(1+|z|_{1}^{M+1})\big{)}.

As in Section 2, a generalization to Hoelder-type conditions (7) in the first component of $g$ is possible. It turns out in Theorem 5.2 that the (pointwise) consistency of $\hat{\theta}_{b}$ can be obtained by posing conditions on the likelihood of the corresponding stationary process which is defined via $L(u,\theta):=\mathbb{E}[\tilde{\ell}_{t}(u,\theta)]$ with $\tilde{\ell}_{t}(u,\theta):=\ell(\tilde{X}_{t}(u),\tilde{Y}_{t-1}(u),\theta)$. Especially if $\ell$ is taken to be of the form (32) with $f_{\varepsilon}$ the standard Gaussian density, the properties of $L(u,\theta)$ are usually well-known from the maximum likelihood theory of the stationary process $X_{t}(\theta)$ and therefore are easy to verify (see also Example 5.5).

To prove consistency, we have to inspect $L_{n,b}(u,\theta)$ which consists of summands of the form $\ell(X_{t,n},Y_{t-1,n},\theta)$. Since $\ell\in\tilde{\mathcal{L}}_{p+1}(M,C)$, these terms behave like polynomials of degree $M+1$ in $X_{t,n}$. We mainly need the law of large numbers Proposition 2.7(ii) and the statements about deterministic bias expansions Proposition 3.3. The conditions therein require Assumption 2.1(S1) with $q=M+1$. Translated to the Markov process setting in this section, we have to assume 4.1(L1), (L4) with $q=M+1$ by the results from Section 4.

Theorem 5.2** (Pointwise and uniform consistency of $\hat{\theta}_{b}$).**

*Let Assumption 2.6 hold. Assume that $\ell\in\tilde{\mathcal{L}}_{p+1}(M,C)$ for some $M\geq 0$. Suppose that Assumption 4.1(L1), (L4) holds with some $1\geq\alpha>0$ and $q=M+1$.

Furthermore suppose that for all $u\in[0,1]$, $\theta_{0}(u)\in\mbox{int}(\Theta)$ is the unique minimizer of $L(u,\theta)$ over $\theta\in\Theta$, where $\Theta\subset\mathbb{R}^{d}$ is a compact set. Then:*

1. (i)

For all $u\in(0,1)$ with $b\to 0$ and $bn\to\infty$:

[TABLE] 2. (ii)

If additionally $q>M+1$ and $b=o(n^{1-\frac{M+1}{q}})$ and $\theta_{0}(\cdot)$ is continuous, we have

[TABLE]

Remark 5.3**.**

Note that in nearly all cases, the conditions of Assumption 4.1(L4) assumed in Theorem 5.2 implicitly impose a Hoelder continuity condition on $\theta_{0}(\cdot)$; see also Example 5.5.

Proof of Theorem 5.2.

(i) For fixed $u\in[0,1]$ and $\theta\in\Theta$, we have $\ell(\cdot,\cdot,\theta)\in\mathcal{L}_{p+1}(M,C_{z})$. Application of Theorem 2.7(ii) (see also Remark 2.8(ii)) leads to

[TABLE]

The function $\theta\mapsto L(u,\theta)$ is continuous since

[TABLE]

It remains to show stochastic equicontinuity of $L_{n,h}(u,\theta)$: Define $h:\mathbb{R}^{p+1}\to\mathbb{R}$, $h(z)=C_{\theta}(1+|z|_{1}^{M+1})$. Fix $\eta>0$. We have

[TABLE]

Obviously, $h\in\mathcal{L}_{p+1}(M,C)$ with some constant $C>0$. Application of Proposition 2.7(ii) to $K/\int K\ \mbox{d}x$ and $h$ (see also Remark 2.8(ii)) yields for all $u\in(0,1)$:

[TABLE]

Choosing $\delta=\frac{\eta}{2c(u)}$ yields

[TABLE]

This gives $\sup_{\theta\in\Theta}|L_{n,b}(u,\theta)-L(u,\theta)|\overset{p}{\to}0$. By standard arguments (cf. Van der Vaart, A.W. (1998), Theorem 5.7), the proof is complete.

To prove (ii), we apply Theorem 2.7(iii) on $\ell(X_{t,n},Y_{t-1,n},\theta)$ with $\tilde{q}=\frac{q}{M+1}>1$ (see also Remark 2.8(ii)) to obtain for each $\theta\in\Theta$ that

[TABLE]

By Proposition 3.3 and the bounded variation of $K$, we have $\sup_{u\in[\frac{b}{2},1-\frac{b}{2}]}|\mathbb{E}L_{n,b}(u,\theta)-L(u,\theta)|=O(b^{\alpha})+O((nb)^{-1})$, which yields

[TABLE]

Similarly we can strengthen (34) to

[TABLE]

Now define $c:=\inf_{u}c(u)>0$ (by continuity of $c(\cdot)$). Choosing $\delta=\frac{\eta}{2c}$ yields

[TABLE]

So we have seen that $\sup_{u\in[\frac{b}{2},1-\frac{b}{2}]}\sup_{\theta\in\Theta}|L_{n,b}(u,\theta)-L(u,\theta)|\overset{p}{\to}0$. Standard arguments give the result (see also the appendix). ∎

We now provide a central limit theorem for $\hat{\theta}_{b}$ including a bias decomposition. Let $\nabla$ denote the derivative with respect to $\theta$. To use a standard Taylor expansion from M-estimation theory, we need the existence of $\nabla\ell\in\tilde{\mathcal{L}}_{p}(M^{\prime},C^{\prime})$ and $\nabla^{2}\ell\in\tilde{\mathcal{L}}_{p}(M^{\prime\prime},C^{\prime\prime})$. To apply the local central limit theorem 2.10 to $\nabla L_{n,b}(u,\theta)$, we additionally need Assumption (S2) with $q=2(M^{\prime}+1)$ which is fulfilled if Assumption 4.1(L2) is valid for $q=2(M^{\prime}+1)$.

Theorem 5.4** (A central limit theorem for $\hat{\theta}_{b}$).**

Additionally to Theorem 5.2(i), suppose that $\ell$ is twice continuously differentiable w.r.t. $\theta$ and

• •

$\nabla\ell\in\tilde{\mathcal{L}}_{p+1}(M^{\prime},C^{\prime})$* for some $M^{\prime}\geq 0$, $\nabla^{2}\ell\in\tilde{\mathcal{L}}_{p+1}(M^{\prime\prime},C^{\prime\prime})$ for some $M^{\prime\prime}\geq 0$,*

• •

Assumption 4.1(L1), (L4) is fulfilled with $q=\max\{2(M^{\prime}+1),M^{\prime\prime}+1\}$ and some $1\geq\alpha^{\prime}>0$, Assumption 4.1(L2) is fulfilled with $q=2(M^{\prime}+1)$.

*Assume that the model is correct in the weak sense that $\mathbb{E}[\nabla\tilde{\ell}(u,\theta_{0}(u))|\mathcal{F}_{t-1}]=0$, i.e. $\nabla\tilde{\ell}_{t}(u,\theta_{0}(u))$ is a martingale difference sequence with respect to $(\mathcal{F}_{t})$. Let $b\to 0$, $nb\to\infty$ and $bn^{1-2\alpha}=o(1)$.

(i) Then we have for $nb^{1+2\alpha^{\prime}}=o(1)$:*

[TABLE]

*where $I(u):=\mathbb{E}[\nabla\tilde{\ell}_{t}(u,\theta_{0}(u))\nabla\tilde{\ell}_{t}(u,\theta_{0}(u))^{\prime}]$ and $V(u):=\nabla^{2}L(u,\theta_{0}(u))$ is assumed to be positive definite.

(ii) If additionally $\nabla\ell$ is continuously differentiable and Assumption 4.1(L3) is fulfilled for $q=M^{\prime}+1$, then we have for $nb^{3}=O(1)$:*

[TABLE]

so the result (35) remains true if $K$ is symmetric.

Proof of Theorem 5.4:.

The conditions on $\nabla^{2}\ell$ imply that $u\mapsto\nabla^{2}L(u,\theta)=\mathbb{E}[\nabla^{2}\tilde{\ell}_{t}(u,\theta)]$ is continuous. Note that by Theorem 2.10, we have

[TABLE]

where \sigma^{2}(u)=\big{\|}\sum_{l=0}^{\infty}P_{0}\nabla\tilde{\ell}_{t}(u,\theta_{0}(u))\big{\|}_{2}^{2}=I(u) by the martingale difference property. Furthermore,

[TABLE]

Proposition 3.3 gives that the first term is $O(\sqrt{nb^{1+2\alpha^{\prime}}})$ in the case of (i). In case of (ii), the first term has the form

[TABLE]

and is $o(\sqrt{nb^{3}})+O((nb)^{-1/2})$ if $K$ is symmetric. Since $\nabla^{2}\ell$ fulfills the same assumptions as $\ell$ in Theorem 5.2, we can mimic its proof and obtain

[TABLE]

By continuity of $\theta\mapsto\nabla^{2}L(u,\theta)$, we obtain for each sequence $\tilde{\theta}_{n}\overset{p}{\to}\theta_{0}(u)$ that

[TABLE]

Standard arguments now give the result. ∎

The results of Theorem 5.4(ii) show that under the existence of derivative processes, one can choose the MSE-optimal rate $b\sim n^{-1/3}$ for the bandwidth, keeping $\hat{\theta}_{b}(u)$ still asymptotically unbiased. This result can be used in several applications, for instance for bootstrapping $X_{t,n}$ via the recursion (31) with estimated errors $\hat{\varepsilon}_{t}=H(X_{t,n},Y_{t-1,n},\hat{\theta}_{b}(t/n))$, $t=p+1,...,n$.

An important special case is the case of Gaussian conditional likelihoods combined with nonlinear autoregressive models. Specific examples for these are given in Example 2.2.

Example 5.5** (Nonlinear autoregressive models).**

In this example we discuss the model $\tilde{G}_{\varepsilon}(y,\theta)=\mu(y,\theta)+\sigma(y,\theta)\varepsilon$, where $\mu,\sigma:\mathbb{R}^{p}\times\Theta\to\mathbb{R}$ satisfy

[TABLE]

with some $\chi\in\mathbb{R}^{p}_{\geq 0}$ with $|\chi|_{1}<1$. Assume that $\mathbb{E}\varepsilon_{0}=0$ and $\mathbb{E}\varepsilon_{0}^{2}=1$ and that $\theta_{0}$ is Hoelder-continuous with exponent $\alpha$. Then Assumption 4.1(L2) is fulfilled with $q=2$.

If we choose $f_{\varepsilon}$ to be the standard Gaussian density, we obtain from (32):

[TABLE]

Furthermore assume that

[TABLE]

Let $\sigma(\cdot)\geq\delta_{\sigma}$ be uniformly bounded from below with some $\delta_{\sigma}>0$. Then $\ell\in\tilde{\mathcal{L}}_{p+1}(1,C)$ with some $C>0$, and Assumption 4.1(L1),(L4) is fulfilled with $q=2$ and $\alpha$ from above.

Fix $u\in[0,1]$. Suppose that

[TABLE]

implies $\theta=\theta_{0}(u)$. Then $\theta\mapsto L(u,\theta)$ has a unique minimum in $\theta=\theta_{0}(u)$ since $\log(x)\leq x-1$ if and only if $x=1$ and $x^{2}\geq 0$ if and only if $x=0$ and, omitting the argument $\tilde{Y}_{t-1}(u)$,

[TABLE]

If additionally $\Theta$ is compact and $\theta_{0}(u)\in\mbox{int}(\Theta)$, the assumptions of Theorem 5.2 are fulfilled and we obtain for $\hat{\theta}_{b}$ defined by (33):

[TABLE]

We now will show asymptotic normality of $\hat{\theta}_{b}$. To keep the presentation simple, we will assume $\sigma(\cdot,\cdot)\equiv 1$, $\mathbb{E}\varepsilon_{0}^{4}<\infty$ and replace $\mathbb{E}\varepsilon_{0}^{2}=1$ by $\mathbb{E}\varepsilon_{0}^{2}=\sigma_{0}^{2}>0$. Note that Assumption 4.1(L2) is fulfilled with $q=4$. Then, omitting the arguments $(y,\theta)$ of $\mu$, we have

[TABLE]

This shows $\mathbb{E}[\nabla\ell(\tilde{X}_{t}(u),\tilde{Y}_{t-1}(u),\theta_{0}(u))|\mathcal{F}_{t-1}]=0$ and $I(u)=\mathbb{E}[\nabla\ell\cdot\nabla\ell^{\prime}]=\sigma_{0}^{2}\mathbb{E}[\nabla\mu\cdot\nabla\mu^{\prime}]=\sigma_{0}^{2}V(u)$ with $V(u):=\nabla^{2}L(u,\theta_{0}(u))$. If additionally

[TABLE]

and similar assumptions are fulfilled for $\nabla^{2}\mu$, then we have $\nabla\ell,\nabla^{2}\ell\in\tilde{\mathcal{L}}_{p+1}(1,C^{\prime})$ with some $C^{\prime}>0$. This shows that all conditions of the first part of Theorem 5.4 are fulfilled and we obtain for $b\to 0$, $nb\to\infty$ and $nb^{3}=o(1)$:

[TABLE]

If additionally, $\mu,\nabla\mu$ and $\theta_{0}$ are continuously differentiable and

[TABLE]

then $\nabla\ell$ is continuously differentiable and Assumption 4.1(L3) is fulfilled with $q=2$. If $K$ is symmetric, all conditions of the second part of Theorem 5.4 are fulfilled and we obtain (40) even if $nb^{3}=O(1)$.

We close this section by using the results of Example 5.5 in a more specific example of the tvExpAR(1) process which is a locally stationary version of the ExpAR(1) process discussed in Jones, D. A. (1978). Up to now, there is no asymptotic theory available for parameter estimators in this model; we show that our theory immediately provides consistency and asymptotic normality of the corresponding maximum likelihood estimator.

Example 5.6** (Maximum likelihood estimation in the tvExpAR(1) process).**

Assume that there exists $\theta_{0}:[0,1]\to\Theta$ (where the image of $\theta_{0}$ is in the interior of $\Theta$) with $\Theta:=\{\theta\in\mathbb{R}:0\leq\theta\leq\rho\}$ and some fixed $\rho>0$, $0<|a_{0}|<1$ such that

[TABLE]

Assume that $\mathbb{E}\varepsilon_{0}=1$, $\mathbb{E}\varepsilon_{0}^{2}=\sigma_{0}^{2}>0$ and $\mathbb{E}\varepsilon_{0}^{4}<\infty$. It is easily seen that this model fulfills the smoothness assumptions (36), (38), (39) and (41) with $\mu(y,\theta):=a_{0}\exp(-\theta y^{2})y$ and $\sigma(\cdot,\cdot)\equiv 1$. Let $\tilde{X}_{t}(u)$ denote the corresponding stationary approximation of $X_{t,n}$. Identifiability of $\theta$ is obtained due to

[TABLE]

since $\mathbb{E}[\exp(-2\rho\tilde{X}_{t}(u)^{2})\tilde{X}_{t}(u)^{6}]=0$ would imply $\tilde{X}_{t}(u)=0$ a.s. which is a contradiction to $\mathbb{E}[\tilde{X}_{t}(u)^{2}]\geq\sigma_{0}^{2}$ which follows from the recursion of $\tilde{X}_{t}(u)$. Let $\hat{\theta}_{b}(u)$ be defined by (33) based on the likelihood (37) and let Assumption 2.6 hold. We obtain for $b\to 0$, $bn\to\infty$:

[TABLE]

and for $nb^{3}=O(1)$:

[TABLE]

where $V(u)=a_{0}^{2}\mathbb{E}[\exp(-2\theta_{0}(u)\tilde{X}_{0}(u)^{2})\tilde{X}_{0}(u)^{6}]$.

6 Concluding Remarks

In this paper, we have made some steps towards a general asymptotic theory for nonlinear locally stationary processes. A key role in our derivations is played by the local stationary approximation, the derivative process, the corresponding Taylor-expansion and the resulting differential calculus.

Just based on this local approximation we were able to prove laws of large numbers, a central limit theorem, and stochastic and deterministic bias approximations - results which have not been proved so far for general locally stationary processes. For example for the global strong law of large numbers we need only the existence of the first order moment of the process. It should be noted that for these results we concluded from local assumptions to global results such as the strong law of large numbers and the central limit theorem. A simulation displayed in Figure 1 shows that the pointwise approximation of $X_{t,n}$ by $\tilde{X}_{t}(u)$ and $\partial_{u}\tilde{X}_{t}(u)$ works quite well.

We also showed that these results can be applied to a general nonlinear time series model with a nonstationary Markov structure which includes several nonlinear models. As another application we derived the asymptotic properties of the maximum likelihood estimator for such processes. The result is proved by applying the differential calculus of the derivative process.

Acknowledgements

We are very grateful to the associate editor and a referee whose comments lead to a considerable improvement of the paper. In particular the observation that in many cases differentiability in $L^{q}$ is sufficient (see Remark 4.3) was pointed out by the referee. We also gratefully acknowledge support by Deutsche Forschungsgemeinschaft through the Research Training Group RTG 1653.

7 Supplement A

This supplement contains the remaining proofs for Section 2, 4 and 5.

7.1 Proofs of Section 2

Let us first cite a Lemma from Dahlhaus, R., and Subba Rao, S. (2006) (Lemma A.1 and A.2) which can be easily generalized to convergence in $L^{1}$:

Lemma 7.1**.**

Assume that $(Y_{t})$ is a stationary and ergodic process with $\mathbb{E}|Y_{1}|<\infty$. Assume that $u\in(0,1)$. Let $b=b_{n}\to 0$ such that $nb_{n}\to\infty$. Then the following convergence holds in $L^{1}$:

[TABLE]

Proof of Theorem 2.7.

(i) Without loss of generality, let us assume $\alpha\leq 1$. For $J\in\mathbb{N}$ and $j=1,...,2^{J}$ define intervals of indices $I_{j,J,n}:=\{t:t/n\in(\frac{j-1}{2^{J}},\frac{j}{2^{J}}]\}$ such that $\bigcup_{j=1}^{2^{J}}I_{j,J,n}=\{1,...,n\}$. For fixed $J\in\mathbb{N}$, we have

[TABLE]

and

[TABLE]

Note that for fixed $J$, by the ergodic theorem for stationary sequences we have for $n\to\infty$:

[TABLE]

By the continuity of $[0,1]\to\mathbb{R},u\mapsto\mathbb{E}\tilde{W}_{0}(u)$, we have

[TABLE]

Finally,

[TABLE]

Thus for all $J\in\mathbb{N}$:

[TABLE]

The limit $J\to\infty$ gives the result.

(ii) To prove the local weak law of large numbers, first note that

[TABLE]

This shows that it is enough to consider the convergence of the sum with the corresponding stationary sequence. From Lemma 7.1, we have that \frac{1}{nb}\sum_{t=1}^{n}K\big{(}\frac{t/n-u}{b}\big{)}\cdot\tilde{X}_{t}(u)\to\mathbb{E}\tilde{X}_{t}(u) holds in $L^{1}$, which finishes the proof.

(iii) Define S_{n}(u):=\sum_{t=1}^{n}K\big{(}\frac{t/n-u}{b}\big{)}\big{(}X_{t,n}-\mathbb{E}X_{t,n}\big{)} and \tilde{S}_{n}(u):=\sum_{t=1}^{n}K\big{(}\frac{t/n-u}{b}\big{)}\big{(}\tilde{X}_{t}(t/n)-\mathbb{E}\tilde{X}_{t}(t/n)\big{)} and $\tilde{S}_{k,n}:=\sum_{t=1}^{k}\tilde{X}_{t}(t/n)$. By partial summation, we have

[TABLE]

Since $K$ is of bounded variation $B_{K}$, we have \sum_{t=1}^{n-1}\big{|}K\big{(}\frac{t/n-u}{b}\big{)}-K\big{(}\frac{(t+1)/n-u}{b}\big{)}\big{|}\leq B_{K} and thus

[TABLE]

The same calculation yields

[TABLE]

First assume $1<q\leq 2$. By using the decomposition $\tilde{X}_{t}(t/n)-\mathbb{E}\tilde{X}_{t}(t/n)=\sum_{l=0}^{\infty}P_{t-l}\tilde{X}_{t}(t/n)$ and applying Doob’s $L^{q}$ maximal inequality (cf. Theorem 5.4.3 in Durrett, R. (2010)), Burkholder’s inequality (cf. Burkholder, D. L. (1988)) and the elementary inequality $(|a_{1}|+|a_{2}|)^{q/2}\leq|a_{1}|^{q/2}+|a_{2}|^{q/2}$, we obtain

[TABLE]

which shows that

[TABLE]

If $q>2$, we use a Nagaev-type inequality from Liu, W., Xiao, H., and Wu, W. B. (2013), Theorem 2(ii) which also holds in our situation as the authors point out in their Section 4. Applying this theorem to $\tilde{S}_{t,n}$ and $-\tilde{S}_{t,n}$, we have for all $x>0$:

[TABLE]

with positive constants $C_{1},C_{2}$ not depending on $n$. Using (42), we obtain

[TABLE]

In case that knowledge of the dependence measure $\delta_{q}^{X_{\cdot,n}}$ of the locally stationary process is available, the approximation of $X_{t,n}$ by $\tilde{X}_{t}(t/n)$ is not necessary and therefore the discussion of the terms $|S_{n}(u)-\tilde{S}_{n}(u)|$ can be omitted. ∎

Proof of Proposition 2.9.

For the proof, we use Theorem 5.46 in Witting, H. and Müller-Funk, U. (1995). Put $S_{k,n}=\sum_{t=1}^{k}X_{t,n}$ and $\tilde{S}_{k,n}:=\sum_{t=1}^{k}\tilde{X}_{t}(t/n)$. Note that by $\alpha>\frac{1}{2}$,

[TABLE]

Put \tilde{S}_{k,n,L}:=\sum_{l=0}^{L-1}\sum_{t=1}^{k}P_{t-l}\tilde{X}_{t}\big{(}\frac{t}{n}\big{)}. Use the abbreviation l.i.m. for $\limsup_{L\to\infty}\limsup_{n\to\infty}$. Because $P_{t-l}\tilde{X}_{t}(t/n)-\mathbb{E}\tilde{X}_{t}(t/n)\to 0$ a.s. and in $L^{1}$ for $l\to\infty$, we have by Doob’s maximal inequality:

[TABLE]

This shows that $S_{\lfloor nu\rfloor,n}/\sqrt{n}$ can be approximated by $\tilde{S}_{\lfloor nu\rfloor,n,L}/\sqrt{n}$. In the case that the dependence measure $\delta_{2}^{X_{\cdot,n}}(l)$ of $X_{t,n}$ is defined and summable in $l$, we can omit the condition $\alpha>\frac{1}{2}$ by using the following argument. Define $S_{k,n,L}:=\sum_{l=0}^{L-1}\sum_{t=1}^{k}P_{t-l}X_{t,n}$. Similarly as above, it can be shown that

[TABLE]

Furthermore, it holds that

[TABLE]

By similar arguments as in the calculation above, we obtain

[TABLE]

which in turn also shows that $S_{\lfloor nu\rfloor,n}/\sqrt{n}$ can be approximated by $\tilde{S}_{\lfloor nu\rfloor,n,L}/\sqrt{n}$.

Now fix $L\in\mathbb{N}$. Define the index-shifted variant of $\tilde{S}_{k,n,L}$ by \hat{S}_{k,n,L}:=\sum_{t=1}^{k}\Big{(}\sum_{l=0}^{L-1}P_{t}\tilde{X}_{t+l}\big{(}\frac{t+l}{n}\big{)}\Big{)}, where $\tilde{X}_{t}(u):=\tilde{X}_{t}(1)$ for $u>1$. For $T=1,...,n$, we have

[TABLE]

Finally, define the martingale differences M_{t,l}:=P_{t}\tilde{X}_{t+l}\big{(}\frac{t+l}{n}\big{)} and the stationary martingale differences M_{l}(u):=P_{0}\tilde{X}_{l}\big{(}u). Note that $M_{t,l}$ has the same distribution as $M_{l}(\frac{t+l}{n})$. We have

[TABLE]

which shows $\frac{1}{\sqrt{n}}\sup_{u\in[0,1]}|\tilde{S}_{\lfloor nu\rfloor,n,L}-\hat{S}_{\lfloor nu\rfloor,n,L}|\overset{p}{\to}0$.

We now investigate the weak convergence of $\hat{S}_{\lfloor nu\rfloor,L}/\sqrt{n}$ with a martingale central limit theorem from Billingsley, P. (2013), Theorem 18.2. Note that $\sum_{l=0}^{L-1}M_{t,l}/\sqrt{n}$ is a martingale difference sequence with respect to $\mathcal{F}_{t}$. By elementary inequalities it can be seen that for each $T=1,...,n$ and each $\varepsilon>0$,

[TABLE]

is bounded by finitely many (dependent on $L$) terms of the form

[TABLE]

where $l,l^{\prime}\in\{0,...,L-1\}$. By using similar techniques as in (43), it can be shown that these converge to 0.

It remains to investigate the behavior of

[TABLE]

for $T=\lfloor sn\rfloor$, $s\in(0,1]$ and $l,l^{\prime}\in\{0,...,L-1\}$. Define $I_{k,K,T}:=\{t:\frac{t}{T}\in(\frac{k-1}{2^{K}},\frac{k}{2^{K}}]\}$, then we have for $K\in\mathbb{N}$:

[TABLE]

which is bounded by $\frac{2^{K}}{T}\sup_{u}\|\tilde{X}_{0}(u)\|_{2}^{2}$. Furthermore, since $\frac{t}{T}\in I_{k,K,T}$ implies $|\frac{t+l}{n}-\frac{k}{2^{K}}s|\leq 2^{-K}+\frac{L}{n}$, we obtain

[TABLE]

where $M_{t,l}(u):=P_{t}\tilde{X}_{t+l}(u)$. Since $\mathbb{E}[M_{t,l}(u)M_{t,l^{\prime}}(u)|\mathcal{F}_{t-1}]$ is ergodic, we have

[TABLE]

In total, performing first $n\to\infty$ and afterwards $K\to\infty$, we obtain

[TABLE]

So we have seen that \{S_{\lfloor nu\rfloor}/\sqrt{n},0\leq u\leq 1\}\overset{d}{\to}\{\int_{0}^{u}\Big{\|}\sum_{l=0}^{L-1}P_{0}\tilde{X}_{l}(v)\Big{\|}_{2}\ \mbox{d}B(v),0\leq u\leq 1\}. By the dominated convergence theorem, \int_{0}^{u}\Big{\|}\sum_{l=0}^{L-1}P_{0}\tilde{X}_{l}(v)\Big{\|}_{2}^{2}\ \mbox{d}v\to\int_{0}^{u}\sigma^{2}(v)\ \mbox{d}v for $L\to\infty$, which completes the proof. ∎

Proof of Theorem 2.10.

Define W_{n,b}:=\frac{1}{\sqrt{nb}}\sum_{t=1}^{n}K\Big{(}\frac{t/n-u}{b}\Big{)}\cdot(X_{t,n}-\mathbb{E}X_{t,n}). Note that

[TABLE]

Since $\|X_{t,n}-\tilde{X}_{t}(t/n)\|_{1}\leq C_{B}n^{-\alpha}$ by assumption, the term above is of order $\sqrt{nb}n^{-\alpha}$. Since $\sum_{k=0}^{\infty}\sup_{u}\delta^{\tilde{X}(u)}_{2}(k)<\infty$, $|K|_{\infty}<\infty$ and $(K_{b}(t/n-u)P_{t-l}\tilde{X}_{t}(t/n))_{t}$ is a martingale difference sequence with respect to $(\mathcal{F}_{t-l})$, we can use the same technique as in the proof of Theorem 2.9 to show that

[TABLE]

Now fix $L\in\mathbb{N}$. Since $K$ is Lipschitz continuous and $\sup_{t}\|\tilde{X}_{t}((t+l)/n)-\tilde{X}_{t}(t/n)\|_{1}\leq C_{B}l^{\alpha}n^{-\alpha}$, it is enough to consider the weak convergence of $\sum_{t=1}^{n}W_{t}(t/n)$, where we define W_{t}(v):=\sum_{l=0}^{L-1}K\Big{(}\frac{t/n-u}{b}\Big{)}P_{t}\tilde{X}_{t+l}(v)/\sqrt{nb}. Note that $W_{t}(t/n)$ is a martingale difference sequence w.r.t. $\mathcal{F}_{t}$. It holds that

[TABLE]

By Lemma 7.1,

[TABLE]

Fix $\varepsilon>0$. The sum $\sum_{t=1}^{n}\mathbb{E}[W_{t}^{2}(t/n)\mathbbm{1}_{\{|W_{t}(t/n)|\geq\varepsilon\}}]$ is bounded by finitely many (dependent on $L$) terms of the form

[TABLE]

which converges to 0 since $\|\sup_{u}|P_{0}\tilde{X}_{l}(u)|\|_{2}<\infty$ by assumption. So we can apply Theorem 18.1. from Billingsley, P. (2013) to obtain

[TABLE]

and thus by Theorem 5.46 in Witting, H. and Müller-Funk, U. (1995),

[TABLE]

∎

7.2 Proofs of Section 4

Here, we prove the results from Section 4. The following lemma from Duflo, M. (1997), Lemma 6.2.10 therein will be used frequently to verify the geometric decay of the difference of recursively defined processes:

Lemma 7.2**.**

Assume that $p>0$ is a positive natural number, $\chi\in\mathbb{R}^{p}_{\geq 0}$ with $|\chi|_{1}<1$ and that there are sequences of real-valued nonnegative numbers $(z_{s})_{s>-p}$, $(\mu_{s})_{s>0}$ which fulfill for all $s=1,2,...$:

[TABLE]

Then there exist constants $\lambda_{0}\in(0,1)$, $C_{\lambda}>0$ only depending on $\chi,p$ such that for all $s=1,2,...$:

[TABLE]

Sometimes we will apply the lemma for $s=0,1,2,...$ instead of $s=1,2,3,...$ .

For the following proofs, recall the abbreviations $Y_{t-1,n}=(X_{t-1,n},...,X_{t-p,n})$ and $\tilde{Y}_{t-1}(u)=(\tilde{X}_{t-1}(u),...,\tilde{X}_{t-p}(u))$. For $y\in\mathbb{R}^{p}$, we will use the abbreviation $G_{\varepsilon,u}(y):=G_{\varepsilon}(y,u)$. Define the random map $R_{\varepsilon,u}(y):=(G_{\varepsilon,u}(y),y_{1},...,y_{p-1})$. Let $X_{n,u}(y)$ be the first element of the vector $H_{n,u}(y):=R_{\varepsilon_{0},u}\circ R_{\varepsilon_{-1},u}\circ...\circ R_{\varepsilon_{-n},u}(y)$, where $n=0,1,2,...$ . For consistency of the following argumentations, define $X_{n,u}(y):=y_{-n}$ for $n=-1,...,-p$. Note that $H_{n,u}(y)_{j}=X_{n-j+1,u}(y)$ (in distribution) for $j=1,...,p$. Let $J_{n,u}(y)$ be defined similarly to $H_{n,u}(y)$ but based on $\varepsilon_{-1},...,\varepsilon_{-n-1}$ instead of $\varepsilon_{0},...,\varepsilon_{-n}$. Note that $X_{n,u}(y)=G_{\varepsilon_{0},u}(J_{n-1,u}(y))$ and that $J_{n-1,u}(y)=H_{n-1,u}(y)=(X_{n-1,u}(y),...,X_{n-p,u}(y))^{\prime}$ holds in distribution.

Proof of Proposition 4.4.

(i) Note that $(|a|+|b|)^{q^{\prime}}\leq|a|^{q^{\prime}}+|b|^{q^{\prime}}$ since $0<q^{\prime}\leq 1$. By (22), we obtain

[TABLE]

By Lemma 7.2, we have with some $C_{\lambda}>0,\lambda_{0}\in(0,1)$ independent of $u\in[0,1]$ that for all $n\in\mathbb{N}$:

[TABLE]

Applying (45) to $y=y_{0}$ and $y^{\prime}=R_{\varepsilon_{-n-1},u}(y_{0})$, we obtain

[TABLE]

By Markov’s inequality and Borel-Cantelli’s lemma, this shows that $(X_{n,u}(y_{0}))_{n\in\mathbb{N}}$ is a Cauchy sequence a.s. and thus has an almost sure limit $\tilde{X}_{0}(u)$ (say). Furthermore, we have

[TABLE]

By Fatou’s lemma,

[TABLE]

since $\sup_{u\in[0,1]}\|G_{\varepsilon_{0}}(y_{0},u)\|_{q}<\infty$ by assumption.

Since $\tilde{X}_{0}(u)$ is $\mathcal{F}_{0}$-measurable, we can write $\tilde{X}_{0}(u)=H(u,\mathcal{F}_{0})$ for some measurable function $H$. By (45), $X_{n,u}(y)$ converges almost surely to the same limit $\tilde{X}_{0}(u)$ for arbitrary $y\in\mathbb{R}^{p}$. This shows a.s. uniqueness among all $\mathcal{F}_{0}$-measurable processes and we can express $\tilde{X}_{t}(u)=H(u,\mathcal{F}_{t})$ a.s. Put $\tilde{X}_{t}^{*0}(u)=H(u,\mathcal{F}_{t}^{*0})$ for $t\in\mathbb{Z}$. Because $\tilde{X}_{t}(u)$ obeys (4), we have for $X_{t}^{*0}(u)=H(u,\mathcal{F}_{t}^{*0})$ by (22):

[TABLE]

By Lemma 7.2, we conclude \big{(}\delta_{q}^{\tilde{X}(u)}(k)\big{)}^{q^{\prime}}=\|\tilde{X}_{t}(u)-\tilde{X}_{t}^{*0}(u)\|_{q}^{q^{\prime}}\leq 2pC_{\lambda}\lambda_{0}^{t}\|\tilde{X}_{0}(u)\|_{q}^{q^{\prime}}.

(ii) Because $X_{0,n}=\tilde{X}_{0}(0)$ by means of (3), the existence and the a.s. uniqueness statement is obvious from Proposition 4.4(i). From (22) and the triangle inequality, we obtain

[TABLE]

Since $\|X_{s,n}\|_{q}^{q^{\prime}}=\|\tilde{X}_{0}(0)\|_{q}^{q^{\prime}}$ for $s\leq 0$, Lemma 7.2 implies

[TABLE]

for all $t=1,...,n$, which gives $\sup_{n\in\mathbb{N}}\sup_{t=1,...,n}\|X_{t,n}\|_{q}^{q^{\prime}}<\infty$. Note that for arbitrary $t\geq 0$, $k\geq 0$, we have by (22):

[TABLE]

Note that $z_{s}:=\|X_{s+(t-k),n}-X_{s+(t-k),n}^{*(t-k)}\|_{q}^{q^{\prime}}=0$ for $s<0$ and furthermore, $z_{0}\leq 2\sup_{n\in\mathbb{N}}\sup_{t=1,...,n}\|X_{t,n}\|_{q}^{q^{\prime}}$. Lemma 7.2 implies

[TABLE]

∎

Proof of Lemma 4.5.

The first inequality (26) is a consequence of

[TABLE]

For the second inequality, note that we have for all $s=1,...,n$:

[TABLE]

Define $z_{s}:=\|X_{s,n}-\tilde{X}_{s}(\frac{s}{n}\vee 0)\|_{q}^{q^{\prime}}$. Note that $z_{s}=0$ for $s\leq 0$ and define $\mu:=C^{q^{\prime}}p^{\alpha q^{\prime}}\frac{|\chi|_{1}}{1-|\chi|_{1}}\cdot n^{-\alpha q^{\prime}}$. In this special case we can calculate the constants from Lemma 7.2 directly, since $z_{s-i_{1}-...-i_{s}}=0$ for $i_{1},...,i_{s}\in\{1,...,p\}$:

[TABLE]

which yields $z_{s}\leq\frac{\mu}{1-|\chi|_{1}}$ and thus

[TABLE]

∎

Proof of Theorem 4.6.

With out loss of generality, we prove the statement for $t=0$. Because of the continuity of $G$, the process $(X_{n,u}(y_{0}))_{u\in[0,1]}$ is continuous and thus a random element of the normed space $(C[0,1],|\cdot|_{\infty})$ where $|\cdot|_{\infty}$ denotes the supremum norm on $[0,1]$. With condition (23) we obtain for two functions $u\mapsto y(u),\tilde{y}(u)$:

[TABLE]

Lemma 7.2 implies that there exist $C_{\lambda}>0$, $0\leq\lambda<1$ such that

[TABLE]

Taking $y(u)=y_{0}$, $\tilde{y}(u)=R_{\varepsilon_{-n-1},u}(y_{0})$, we conclude

[TABLE]

Markov’s inequality and Borel-Cantelli’s lemma implies that the sequence $(X_{n,u}(y_{0}))_{u\in[0,1]}$, $n\in\mathbb{N}$ of elements of $C[0,1]$ is a Cauchy sequence in $(C[0,1],|\cdot|_{\infty})$ almost surely. Since this space is complete, there exists a continuous limit $\hat{X}_{0}=(\hat{X}_{0}(u))_{u\in[0,1]}$. It was already shown in the proof of Proposition 4.4 that $X_{n,u}(y_{0})\to\tilde{X}_{0}(u)$ a.s. for fixed $u\in[0,1]$. This implies that $\hat{X}_{0}$ is a continuous modification of $(\tilde{X}_{0}(u))_{u\in[0,1]}$. By (47), we have

[TABLE]

Because for $M\in\mathbb{N}$, $M\wedge\sup_{u\in[0,1]}|\cdot|$ is a bounded and continuous functional, we obtain \big{\|}M\wedge\sup_{u\in[0,1]}|\hat{X}_{0}(u)|\big{\|}_{q}\leq D. The monotone convergence theorem implies $\sup_{u\in[0,1]}|\hat{X}_{t}(u)|\in L^{q}$. ∎

Proposition 7.3**.**

In the situation of Theorem 4.6, instead of (23) assume that $x\mapsto G_{\varepsilon}(x,u)$ is differentiable for all $\varepsilon,u$ and that for all $u_{0}\in[0,1]$,

[TABLE]

and

[TABLE]

Then the results of Theorem 4.6 are still valid.

Proof of Proposition 7.3.

For fixed $u_{0}\in[0,1]$, the fundamental theorem of calculus gives

[TABLE]

The first term is bounded in absolute value by $\sup_{x}|\partial_{1}G_{\varepsilon_{0}}(x,u)-\partial_{1}G_{\varepsilon_{0}}(x,u_{0})|_{1}\cdot|y-y^{\prime}|_{\infty}$. Since $|\chi|_{1}<1$, we can assume w.l.o.g. that $\chi_{j}>0$ for all $j=1,...,p$ (if for instance $\chi_{1}=0$, one can define $\chi^{\prime}:=\chi+(1-|\chi|_{1}/2,0,...,0)$ which still fulfills $|\chi^{\prime}|_{1}<1$). Now choose $\beta>1$ such that $\beta|\chi|_{1}<1$, and define $\chi^{\prime}:=\delta\chi$ for some $\delta>0$. We have $|y-y^{\prime}|_{\infty}\leq\frac{1}{\min(\chi^{\prime})}|y-y^{\prime}|_{\chi^{\prime},q^{\prime}}$. For $\delta$ small enough, we have

[TABLE]

Fix $t\in\mathbb{Z}$. Partitioning of $[0,1]$ into (overlapping) closed intervals $I_{1},...,I_{K}$ of length at most $\delta$ and applying Theorem 4.6 on each of these intervals $I_{k}$, $k=1,...,K$ provides the existence of a continuous modification of $(\hat{X}_{t}^{(k)}(u))_{u\in I_{k}}$ of $(\tilde{X}_{t}(u))_{u\in I_{k}}$ on each of these subintervals with $\sup_{u\in I_{k}}|\hat{X}_{t}^{(k)}(u)|\in L^{q}$. For fixed $k,k^{\prime}\in\{1,...,K\}$ with $I_{k}\cap I_{k^{\prime}}\not=\emptyset$ the continuous processes $(\hat{X}_{t}^{(k)}(u))_{u\in I_{k}}$, $(\hat{X}_{t}^{(k^{\prime})}(u))_{u\in I_{k^{\prime}}}$ are a.s. equal on $I_{k}\cap I_{k^{\prime}}$ which ensures continuity of a process $(\hat{X}_{t}(u))_{u\in[0,1]}$ which is assembled from $(\hat{X}_{t}^{(k)}(u))_{u\in I_{k}}$, $k=1,...,K$ and thus a modification of $(\tilde{X}_{t}(u))_{u\in[0,1]}$. ∎

Proof of Theorem 4.8.

(i) Note that Assumption 4.1(L2),(L3) imply 4.1(L1) and (LABEL:DP_cond_3). We will only use these conditions for the following proof. Since the process $\tilde{X}_{t}(u)$ is already known to exist, we are able to define a new recursion function based on $\tilde{X}_{t}(u)$. For $y\in\mathbb{R}^{p}$, define the random map $\hat{G}_{t}(y,u):=\langle\partial_{1}G_{\varepsilon_{t}}(\tilde{Y}_{t-1}(u),u),y\rangle+\partial_{2}G_{\varepsilon_{t}}(\tilde{Y}_{t-1}(u),u)$ and $\hat{R}_{t,u}(y):=(\hat{G}_{t}(y,u),y_{1},...,y_{p-1})$, and let $D_{t,n,u}(y)$ be the first element of $\hat{R}_{t,u}\circ\hat{R}_{t-1,u}\circ...\circ\hat{R}_{t-n,u}(y)$ for $n\in\mathbb{N}$. For $y,y^{\prime}\in\mathbb{R}^{p}$, (22) and Fatou’s lemma imply

[TABLE]

Similar to the proof of Proposition 4.4, we obtain $C_{\lambda}>0,\lambda_{0}\in(0,1)$ with

[TABLE]

Applying this to $y=y_{0}$ and $y^{\prime}=\hat{R}_{t-n-1,u}(y_{0})$ we obtain

[TABLE]

which is finite by (LABEL:DP_cond_3) and (49). This implies that $D_{0,n,u}(y_{0})$ converges a.s. to some limit $D_{0}(u)$, say. Because $\tilde{X}_{k}(u)\in\mathcal{F}_{k}$ ($k\in\mathbb{Z}$), it is obvious that $D_{0}(u)$ is $\mathcal{F}_{0}$-measurable and therefore has a representation $D_{0}(u)=\hat{H}(u,\mathcal{F}_{0})$. The rest of the proof is the same as in Proposition 4.4(i).

(ii) Because of the continuous differentiability of $G$, the process $(X_{n,u}(y_{0}))_{u\in[0,1]}$ is a random element of $(C^{1}[0,1],|\cdot|_{C^{1}})$, where $\|f\|_{C^{1}}=|f|_{\infty}+|f^{\prime}|_{\infty}$ and $|\cdot|_{\infty}$ denotes the supremum norm on $[0,1]$.

We will only consider the case that Assumption 4.1(L3)(a) is fulfilled. In the case of Assumption 4.1(L3)(b), one can set $\tilde{q}=q$ in the following with obvious changes in the proofs.

Define $\tilde{q}:=q/2$ and $\tilde{q}^{\prime}:=\min\{\tilde{q},1\}$. Let $u\mapsto y_{1}(u),y_{2}(u)\in\mathbb{R}^{p}$ be two differentiable functions (for brevity, we will omit the argument $u$ in the following). Because of $X_{n,u}(y)=G_{\varepsilon_{0},u}(J_{n-1,u}(y))$, we have:

[TABLE]

This shows (use similar techniques as in (49)):

[TABLE]

The third term is finite by assumption (follows from (24) and $\|\sup_{u}|\partial_{2}G_{\varepsilon_{0}}(y_{0},u)|\|_{q}<\infty$). In the proof of Theorem 4.6 it was shown that there exist $C_{\lambda}^{\prime}>0$, $0\leq\lambda_{0}^{\prime}<1$ such that for all $n\in\mathbb{N}$: $\|\sup_{u}|X_{n,u}(y_{1})|\|_{q}^{q^{\prime}}\leq D(y_{1})^{q^{\prime}}$ with

[TABLE]

see (48). Since $|\chi|_{1}<1$, Lemma 7.2 and (50) imply that there exist $C_{\lambda}>0$, $0\leq\lambda_{0}<1$ such that for all $n\in\mathbb{N}$:

[TABLE]

Using the triangle inequality, we obtain

[TABLE]

Condition (24) and the result (46) from the proof of Theorem 4.6 (use $\tilde{C}_{\lambda}$, $\tilde{\lambda}_{0}$ for the result therein) implies

[TABLE]

Using Assumption 4.1(L3)(a), a similar technique as in (49) gives

[TABLE]

By the Cauchy-Schwarz inequality, we have

[TABLE]

Finally we have shown that exists a constant $C(y_{1})>0$ such that

[TABLE]

Lemma 7.2 implies that there exist constants $C_{\lambda}>0$, $\lambda_{0}\in(0,1)$ such that for $n\in\mathbb{N}$:

[TABLE]

Taking $y_{1}(u)\equiv y_{0}$, $y_{2}(u)=R_{\varepsilon_{0}}(y_{0},u)$ and using the inequalities

[TABLE]

and $\|\sup_{u}|y_{1}-y_{2}|_{1}\|_{\tilde{q}}\leq\|\sup_{u}|y_{0}-R_{\varepsilon_{0}}(y_{0},u)|_{1}\|_{q}<\infty$ by assumption, we obtain that for all $n\in\mathbb{N}$:

[TABLE]

with $0<\hat{\lambda}_{0}:=\max(\lambda_{0},\tilde{\lambda}_{0}^{\tilde{q}^{\prime}/q^{\prime}})<1$ and some constant $\hat{C}_{\lambda}(y_{0})>0$. Together with the result (47), Markov’s inequality and Borel-Cantelli’s lemma, we obtain that the sequence $(X_{n,u}(y_{0}))_{u\in[0,1]}$, $n\in\mathbb{N}$ of elements of $C^{1}[0,1]$ is a Cauchy sequence in $(C^{1}[0,1],|\cdot|_{C^{1}})$ almost surely. Since this space is complete, there exists a continuously differentiable limit $\hat{X}_{0}=(\hat{X}_{0}(u))_{u\in[0,1]}$. Because $\hat{X}_{0}$ is $\mathcal{F}_{0}$-measurable, there exists a measurable function $\hat{H}=(\hat{H}(u,\cdot))_{u\in[0,1]}:\mathbb{R}^{\mathbb{N}}\to C^{1}[0,1]$ such that $u\mapsto\hat{H}(u,z)$ is continuously differentiable for all $z\in\mathbb{R}^{\mathbb{N}}$. For arbitrary $t\in\mathbb{Z}$, we may define $\partial_{u}\hat{X}_{t}(u):=\partial_{u}\hat{H}(u,\mathcal{F}_{t})$. The process $X_{t,n,u}(y)$ defined similarly as $X_{n,u}(y)$ but with $\varepsilon_{0},...,\varepsilon_{-n}$ replaced by $\varepsilon_{t},...,\varepsilon_{t-n}$ has the same distributional properties as $X_{n,u}(y)$ and therefore $X_{t,n,u}(y)\to\hat{H}(u,\mathcal{F}_{t})$ a.s. and $\partial_{u}X_{t,n,u}(y)\to\partial_{u}\hat{H}(u,\mathcal{F}_{t})$ a.s. By construction it holds that

[TABLE]

and

[TABLE]

thus we obtain for $n\to\infty$ that $\hat{X}_{t}(u)$ fulfills (4) and $\partial_{u}\hat{X}_{t}(u)$ fulfills (28) a.s. for all $t\in\mathbb{Z}$. Since (4), (28) only allow for a.s. unique solutions, we conclude that $(\hat{X}_{t}(u))_{u\in[0,1]}$ is a continuously differentiable modification of $(\tilde{X}_{t}(u))_{u\in[0,1]}$ and $(\partial_{u}\hat{X}_{t}(u))_{u\in[0,1]}$ is a continuous modification of $(D_{t}(u))_{u\in[0,1]}$.

The uniform convergence $\sup_{u}|\partial_{u}X_{n,u}(y_{0})-\partial_{u}\hat{X}_{0}(u)|\to 0$ together with Fatou’s lemma and (51) implies $\sup_{u}|\partial_{u}\hat{X}_{0}(u)|\in L^{q}$. ∎

Proof of Lemma 4.11.

Define $\tilde{q}:=q/2$ and $\tilde{q}^{\prime}:=\min\{\tilde{q},1\}$. Let $u,u^{\prime}\in[0,1]$. Because $\partial_{u}\tilde{X}_{t}(u)$ obeys (28), we have by the Cauchy Schwarz inequality:

[TABLE]

(24) and (29) give

[TABLE]

Similar results are obtained for the first term in (53). Note that $\|\sup_{u}|\partial_{u}\tilde{X}_{t}(u)|\|_{q}\leq M$ with some $M>0$ by Theorem 4.8. The conditions of Lemma 4.5 are fulfilled for $\alpha=1$, alternatively it can be seen directly that

[TABLE]

A similar technique as in (49) applied to the second summand in (53) implies the inequality \big{\|}\langle\partial_{1}G_{\varepsilon_{t}}(\tilde{Y}_{t-1}(u^{\prime}),u^{\prime}),\partial_{u}\tilde{X}_{t-1}(u)-\partial_{u}\tilde{X}_{t-1}(u^{\prime})\rangle\big{\|}_{\tilde{q}}^{\tilde{q}^{\prime}}\leq|\chi|_{1}\|\partial_{u}\tilde{X}_{t}(u)-\partial_{u}\tilde{X}_{t}(u^{\prime})\|_{\tilde{q}}^{\tilde{q}^{\prime}}. We finally obtain

[TABLE]

which gives the result since $|\chi|_{1}<1$. ∎

7.3 Proofs of Section 5

Proof of Theorem 5.2, uniform convergence of $\hat{\theta}_{b}$.

Since a sequence converges in probability to some random variable $Z$ if each subsequence has a further subsequence that converges almost surely towards $Z$, we may assume w.l.o.g. that

[TABLE]

Since $\theta_{0}$ is continuous and $\theta_{0}(u)\in\mbox{int}(\Theta)$ for all $u\in[0,1]$, the whole curve $\theta_{0}$ has a positive $|\cdot|_{1}$-distance $c_{min}:=\inf_{u\in[0,1]}\mbox{dist}(\theta_{0}(u),\partial\Theta)>0$ to the boundary $\partial\Theta$ of $\Theta$. Choose $\varepsilon\in(0,c_{min})$ arbitrarily. For each $u\in D_{u}=D_{u}(n):=[\frac{b}{2},1-\frac{b}{2}]$, define $\Theta(u,\varepsilon):=\{\theta\in\Theta:|\theta-\theta_{0}(u)|_{1}<\varepsilon\}\not=\emptyset$ (nonempty since $\theta_{0}(u)$ is in the interior of $\Theta$ by assumption). Define

[TABLE]

Here, $\theta^{*}(u)$ does not need to be unique, but we choose one of the possible values. Because $\Theta\cap\Theta(u,\varepsilon)^{c}$ is compact, there has to exist at least one. Because $\theta_{0}(u)$ is the unique minimum of $\theta\mapsto L(u,\theta)$ over $\Theta$, there exists $\delta(u)>0$ such that

[TABLE]

It holds that $\delta:=\inf_{u\in[0,1]}\delta(u)>0$. Otherwise, because of the compactness of $[0,1]$, there would exist a sequence $(u_{n})\subset[0,1]$ with $u_{n}\to u^{*}\in[0,1]$ and $\delta(u_{n})\to 0$. By the continuity of $L$, $\theta_{0}$ and $u\mapsto\inf_{\theta\in\Theta\cap\Theta(u,\varepsilon)^{c}}L(u,\theta)$ (use Berge’s Maximum theorem and the fact that $u\mapsto\Theta\cap\Theta(u,\varepsilon)^{c}$ is a continuous set function) this would imply

[TABLE]

which is a contradiction to the fact that $\theta_{0}(u^{*})\in\Theta(u^{*},\delta)$ is the unique minimum of $L(u^{*},\theta)$. By (54), we may choose $N\in\mathbb{N}$ such that for all $n\geq N$, $\sup_{u\in D_{u}}\sup_{\theta\in\Theta}|L_{n,b}(u,\theta)-L(u,\theta)|<\frac{\delta}{2}$. Now suppose that for some $n\geq N$, $\sup_{u\in D_{u}}|\hat{\theta}_{b}(u)-\theta_{0}(u)|_{1}\geq\varepsilon$. Then we have for some $u\in D_{u}$ that

[TABLE]

which is a contradiction to the extremal property of $\hat{\theta}_{b}(u)$. ∎

8 Supplement B

This supplement contains another counterexample where Assumption 2.3(M1) is satisfied but not (M2).

Let $X_{t,n}=\sum_{k=1}^{\infty}a_{t,n}(k)\varepsilon_{t-k}$ be a linear process and $\tilde{X}_{t}(u)=\sum_{k=1}^{\infty}a(u,k)\varepsilon_{t-k}$ the corresponding stationary approximation, where $\varepsilon_{i}$ are i.i.d., $\alpha>0$, $a_{t,n}(k)=\frac{1+\frac{k}{n}}{k^{2+\alpha}+t/n}$ and $a(u,k)=\frac{1}{k^{2+\alpha}+u}$. Then we have

[TABLE]

which ensures $\|X_{t,n}-\tilde{X}_{t}(t/n)\|_{q}\leq n^{-1}\|\varepsilon_{0}\|_{q}\sum_{k=1}^{\infty}k^{-2}$ and $\|\tilde{X}_{t}(u)-\tilde{X}_{t}(u^{\prime})\|_{q}\leq|u-u^{\prime}|\|\varepsilon_{0}\|_{q}\sum_{k=1}^{\infty}k^{-6}$ for $q\geq 1$. However, the processes show different behavior for the dependence measure,

[TABLE]

If we choose more specifically $\varepsilon_{0}\overset{\mbox{iid}}{\sim}N(0,1)$ and $\alpha=0$, then clearly $\|\tilde{X}_{t}(u)\|_{2}<\infty$ exists and for $X_{t,n}$ we have

[TABLE]

which shows $\sup_{t,n}\|X_{t,n}\|_{2}^{2}<\infty$ and \|X_{t,n}-\tilde{X}_{t}(t/n)\|_{2}\leq n^{-1}\big{(}\sum_{k=1}^{\infty}k^{-2}\big{)}^{1/2}, but

[TABLE]

i.e. $\sum_{k=1}^{\infty}\delta_{2}^{X_{\cdot,n}}(k)=\infty$ but $\sum_{k=1}^{\infty}\delta_{2}^{\tilde{X}(u)}(k)<\infty.$