Consistent nonparametric change point detection combining CUSUM and marked empirical processes

TL;DR

This paper introduces a new nonparametric change point detection method for weakly dependent time series, combining CUSUM and empirical processes, with proven convergence and bootstrap applicability.

Contribution

It develops a novel change point test based on a modified CUSUM and empirical process, with proven weak convergence and a simple limiting distribution.

Findings

The test has a simple limiting distribution under the null hypothesis.

The procedure is consistent and applicable with bootstrap methods.

It effectively detects changes in the conditional mean function.

Abstract

A weakly dependent time series regression model with multivariate covariates and univariate observations is considered, for which we develop a procedure to detect whether the nonparametric conditional mean function is stable in time against change point alternatives. Our proposal is based on a modified CUSUM type test procedure, which uses a sequential marked empirical process of residuals. We show weak convergence of the considered process to a centered Gaussian process under the null hypothesis of no change in the mean function and a stationarity assumption. This requires some sophisticated arguments for sequential empirical processes of weakly dependent variables. As a consequence we obtain convergence of Kolmogorov-Smirnov and Cram\'er-von Mises type test statistics. The proposed procedure acquires a very simple limiting distribution and nice consistency properties, features from which related tests are lacking. We moreover suggest a bootstrap version of the procedure and discuss its applicability in the case of unstable variances.

Tables

Table 1. Table 1: Rejection frequencies in model 3

			under $H_0$					under $H_1$
$t_0$	$n$		$T_{n1}^{\ast }$	$T_{n2}^{\ast }$	$KS$	$CM$		$T_{n1}^{\ast }$	$T_{n2}^{\ast }$	$KS$	$CM$
$0.25$	$100$		$0.030$	$0.046$	$0.030$	$0.054$		$0.286$	$0.270$	$0.192$	$0.168$
	$300$		$0.068$	$0.064$	$0.080$	$0.052$		$0.652$	$0.644$	$0.248$	$0.172$
	$500$		$0.060$	$0.052$	$0.058$	$0.046$		$0.878$	$0.868$	$0.264$	$0.194$
$0.50$	$100$		$0.068$	$0.048$	$0.068$	$0.056$		$0.420$	$0.438$	$0.316$	$0.256$
	$300$		$0.066$	$0.050$	$0.056$	$0.046$		$0.868$	$0.894$	$0.378$	$0.292$
	$500$		$0.046$	$0.040$	$0.058$	$0.040$		$0.994$	$0.996$	$0.434$	$0.324$
$0.75$	$100$		$0.060$	$0.056$	$0.072$	$0.070$		$0.404$	$0.388$	$0.332$	$0.266$
	$300$		$0.048$	$0.048$	$0.050$	$0.056$		$0.830$	$0.848$	$0.382$	$0.250$
	$500$		$0.034$	$0.040$	$0.046$	$0.056$		$0.986$	$0.988$	$0.350$	$0.202$

Table 2. Table 2: Rejection frequencies in model 4

			under $H_0$					under $H_1$
model	$n$		$T_{n1}^{\ast }$	$T_{n2}^{\ast }$	$KS$	$CM$		$T_{n1}^{\ast }$	$T_{n2}^{\ast }$	$KS$	$CM$
AR(2)	$100$		$0.082$	$0.068$	$0.082$	$0.074$		$0.124$	$0.110$	$0.080$	$0.070$
	$300$		$0.064$	$0.070$	$0.054$	$0.048$		$0.284$	$0.308$	$0.096$	$0.070$
	$500$		$0.076$	$0.058$	$0.068$	$0.060$		$0.480$	$0.532$	$0.098$	$0.070$
AR(2)-	$100$		$0.076$	$0.060$	$0.094$	$0.068$		$0.098$	$0.106$	$0.070$	$0.058$
ARCH(1)	$300$		$0.084$	$0.098$	$0.086$	$0.096$		$0.252$	$0.282$	$0.088$	$0.074$
	$500$		$0.098$	$0.078$	$0.080$	$0.074$		$0.476$	$0.484$	$0.120$	$0.078$
AR(2)-	$100$		$0.076$	$0.064$	$0.064$	$0.044$		$0.096$	$0.104$	$0.072$	$0.050$
ARCH(2)	$300$		$0.100$	$0.082$	$0.092$	$0.076$		$0.226$	$0.236$	$0.108$	$0.074$
	$500$		$0.082$	$0.068$	$0.076$	$0.056$		$0.392$	$0.420$	$0.094$	$0.068$

Equations

$$H_0:m_t({\mathbf{X}}_t)=m({\mathbf{X}}_t)\text{ a.s. for all }t=1,\dots ,n$$

$$Y_t=m_t({\mathbf{X}}_t)+U_t,t\in \mathbb{Z}.$$

$$H_0:m_t(\cdot )=m(\cdot ),t=1,\dots ,n,$$

$${\hat{T}}_n(s,\mathbf{z})=\frac{1}{\sqrt{n}}\sum _{i=1}^{\lfloor ns\rfloor }(Y_i-{\hat{m}}_n({\mathbf{X}}_i)){\omega }_n({\mathbf{X}}_i)I\{{\mathbf{X}}_i\le \mathbf{z}\},$$

$${\hat{m}}_n(\mathbf{x})$$

$$\frac{1}{\sqrt{n}}\sum _{i=1}^{\lfloor ns\rfloor }(Y_i-{\hat{m}}_n({\mathbf{X}}_i)){\hat{f}}_n({\mathbf{X}}_i)w({\mathbf{X}}_i),$$

$$p_n$$

$$0<q_n$$

$$\left(\sqrt{\frac{\log n}{n{h}_n^{d+2(l+1)}}}+h_n^r{p}_n\right)p_n^{l+1}{\delta }_n^{l+2}=O(1),$$

$$\left(\sqrt{\frac{\log n}{n{h}_n^{d+2(l+1)}}}+h_n^r{p}_n\right)p_n^{l+\eta }{q}_n{\delta }_n^{l+1+\eta }=o(1).$$

$$\frac{(\log n{)}^{3+\frac{d}{l+\eta }}}{\sqrt{n^{1-\frac{d}{l+\eta }}{h}_n^d}}{q}_n^2{\delta }_n^2=o(1),\frac{\log h_n}{\sqrt{n{h}_n^d}}=o(1),\sqrt{n}{h}_n^r{p}_n{q}_n=o(1),(\log n{)}^3{h}_n{q}_n^2=o(1).$$

$${\hat{T}}_n(s,\mathbf{z})$$

$$A_{n1}(s,\mathbf{z}):=$$

$$A_{n2}(s,\mathbf{z}):=$$

$$T_n(s,\mathbf{z})=\frac{1}{\sqrt{n}}\sum _{i=1}^{\lfloor ns\rfloor }{U}_{i}I\{{\mathbf{X}}_i\le \mathbf{z}\},s\in [0,1],\mathbf{z}\in {\mathbb{R}}^d.$$

$$A_{n1}(s,\mathbf{z})$$

$$\frac{s}{\sqrt{n}}\sum _{i=1}^n{\int }_{(-\infty ,\mathbf{z}]}\left(m(\mathbf{y})-m({\mathbf{X}}_i)\right)K_{h_n}(\mathbf{y}-{\mathbf{X}}_i){\omega }_n(\mathbf{y})\frac{f(\mathbf{y})}{{\hat{f}}_n(\mathbf{y})}d\mathbf{y},$$

$$-\frac{s}{\sqrt{n}}\sum _{i=1}^n{U}_i{\int }_{(-\infty ,\mathbf{z}]}{K}_{h_n}(\mathbf{y}-{\mathbf{X}}_i){\omega }_n(\mathbf{y})\frac{f(\mathbf{y})}{{\hat{f}}_n(\mathbf{y})}d\mathbf{y}$$

$$A_{n1}(s,\mathbf{z})=-s{T}_n(1,\mathbf{z})+o_P(1)$$

$${\hat{T}}_n(s,\mathbf{z})=T_n(s,\mathbf{z})-s{T}_n(1,\mathbf{z})+o_P(1),$$

$$\displaystyle\operatorname{Cov}\big{(}G(s_{1},\bm{z}_{1}),G(s_{2},\bm{z}_{2})\big{)}=(s_{1}\wedge s_{2})\Sigma(\bm{z}_{1}\wedge\bm{z}_{2})$$

$$\displaystyle\operatorname{Cov}\big{(}G_{0}(s_{1},\bm{z}_{1}),G_{0}(s_{2},\bm{z}_{2})\big{)}=(s_{1}\wedge s_{2}-s_{1}s_{2})\Sigma(\bm{z}_{1}\wedge\bm{z}_{2})$$

$$T_{n1}$$

$$T_{n3}$$

$$\frac{T_{n1}}{{\hat{c}}_n^{1/2}}$$

$$\frac{T_{n3}}{{\hat{c}}_n^2}$$

$${\tilde{T}}_{n3}$$

$$\hat{\sigma}_{n}^{2}(x)=\dfrac{\sum_{j=1}^{n}K\Big{(}\frac{x-X_{j}}{h_{n}}\Big{)}(Y_{j}-\hat{m}_{n}(x))^{2}}{\sum_{j=1}^{n}K\Big{(}\frac{x-X_{j}}{h_{n}}\Big{)}}.$$

$$Y_{n,t}=m_{n,t}({\mathbf{X}}_{n,t})+U_{n,t},t=1,\dots ,n,$$

$$H_1:\exists s_0\in (0,1):m_{n,t}(\cdot )=\{\begin{array}{ll}{m}_{(1)}(\cdot ),& t=1,\dots ,\lfloor n{s}_0\rfloor \\ m_{(2)}(\cdot ),& t=\lfloor n{s}_0\rfloor +1,\dots ,n\end{array}$$

$$\underset{n\to \infty }{\lim }\frac{1}{n}\sum _{t=1}^{\lfloor ns\rfloor }{f}_{n,t}(\mathbf{x})={\bar{f}}^{(s)}(\mathbf{x}),\forall \mathbf{x}\in {\mathbb{R}}^d.$$

$$\underset{\mathbf{x}\in {\mathbf{J}}_n}{\sup }\left|{\hat{m}}_n(\mathbf{x})-{\bar{m}}_n(\mathbf{x})\right|=o_P(1),$$

Full text

Consistent nonparametric change point detection combining CUSUM and marked empirical processes

Maria Mohr and Natalie Neumeyer

Department of Mathematics, University of Hamburg

A weakly dependent time series regression model with multivariate covariates and univariate observations is considered, for which we develop a procedure to detect whether the nonparametric conditional mean function is stable in time against change point alternatives. Our proposal is based on a modified CUSUM type test procedure, which uses a sequential marked empirical process of residuals. We show weak convergence of the considered process to a centered Gaussian process under the null hypothesis of no change in the mean function and a stationarity assumption. This requires some sophisticated arguments for sequential empirical processes of weakly dependent variables. As a consequence we obtain convergence of Kolmogorov-Smirnov and Cramér-von Mises type test statistics. The proposed procedure acquires a very simple limiting distribution and nice consistency properties, features from which related tests are lacking. We moreover suggest a bootstrap version of the procedure and discuss its applicability in the case of unstable variances.

Key words: bootstrap, change point detection, cumulative sums, distribution-free test, heteroscedasticity, kernel estimation, nonparametric regression, sequential empirical process

AMS 2010 Classification: Primary 62M10, Secondary 62G08, 62G09, 62G10

1 Introduction

Assume a finite sequence $(\bm{X}_{t},Y_{t})$, $t=1,\dots,n$, of a weakly dependent $\mathbb{R}^{d}\times\mathbb{R}$-valued time series has been observed. Here, we interpret $\bm{X}_{t}$ as a covariate (which may contain past values of the process) and it is assumed that the conditional expectation of the observation $Y_{t}$, given $\bm{X}_{t}$ and all past values of the time series, does only depend on the covariate $\bm{X}_{t}$, and thus is a function $m_{t}(\bm{X}_{t})$. We do not impose any parametric structure on the regression function. For inference on the time series it is of importance whether the regression function is time dependent or not, i.e. the hypothesis

[TABLE]

(for some not further specified function $m$) should be tested against structural changes over time such as change point alternatives.

Literature on such tests for nonparametric regression functions is rare in the time series context. Both Hidalgo (1995) and Honda (1997) suggested CUSUM tests for change points in the regression function in nonparametric time series regression models with strictly stationary and absolutely regular data. Su and Xiao (2008) extended these tests to strongly mixing and not necessarily stationary processes, allowing for heteroscedasticity, while Su and White (2010) proposed change point tests in partially linear time series models. Vogt (2015) constructed a kernel-based $L_{2}$-test for structural change in the regression function in time-varying nonparametric regression models with locally stationary regressors.

We will combine the CUSUM approach as considered by Hidalgo (1995), Honda (1997), and Su and Xiao (2008) with a marked empirical process approach. Marked empirical processes have been suggested in a seminal paper by Stute (1997) for lack-of-fit testing in nonparametric regression models with i.i.d. data. Since then they have been widely used for hypothesis testing in regression models, see Koul and Stute (1999) and Delgado and Manteiga (2001), among many others. A marked empirical process approach has been applied by Burke and Bewa (2013) for change point detection in an i.i.d. setting. In contrast to our approach they use a process of observations instead of residuals with a very complicated limit distribution, whereas we obtain a simple limit distribution and even asymptotically distribution-free tests in the case of one-dimensional covariates. To this end we show weak convergence of a sequential marked empirical process of residuals under the null hypothesis. We further demonstrate consistency under fixed alternatives of one structural break in the regression function at some time point $\lfloor ns_{0}\rfloor$ for $n\to\infty$.

Moreover we suggest a wild bootstrap version of our test that can be applied to detect changes in the mean function in the case of stable variances (as alternative to using the asymptotic distribution, e.g. for multivariate covariates) as well as in the case of non-stable variances. Wild bootstrap was first introduced by Wu (1986) and Liu (1988) for linear regression with heteroscedasticity. It was used in time series context by Kreiß (1997) and Hafner and Herwartz (2000), among others. The bootstrap version of our test can detect changes in the conditional mean function, even when the conditional variance function is also not stable, but – as desired – the test does not react sensitive to the unstable variance. If no change in the mean function is detected, a test for change in the variance function can be applied, which assumes a stable mean function. The latter approach will be considered in detail in a forthcoming manuscript. Most literature assumes stationary variances of the error terms (unconditional or conditional) when testing for changes in regression. However, as Wu (2016) pointed out, non-stationary variances can occur and will most likely result in misleading inferences when not taken into account. Although this is a legitimate concern, not many results are available that deal with non-stationary variances. The CUSUM test by Su and Xiao (2008) allows for breaks in the conditional variance function. But their procedure does only seem to work for fixed breaks that do not depend on the sample size, whereas we consider changes of the variance in some $\lfloor nt_{0}\rfloor$ for $n\to\infty$. There are some approaches for testing for parameter stability in parametric time series models that consider unstable variances, see Pitarakis (2004), Perron and Zhou (2008), Kristensen (2012), Cai (2007), Xu (2015) and Wu (2016). But all the settings considered do not fit into our framework as they either do not allow for autoregression models, by assuming stationarity of the regressor variables under the null, or they do not cover heteroscedastic effects. More precisely if heteroscedasticity is considered, variance instabilities are not modeled in the conditional variance function but as a time-varying constant.

The paper is organized as follows. In section 2 we present the model and the sequential marked empirical process, on which the test statistics are based. Further the assumptions are listed. In section 3 we consider the limit distribution under the null hypothesis as well as consistency under the fixed alternative of one change point. The wild bootstrap version of the procedure is discussed in section 4, whereas simulations and a real data example are presented in section 5. Section 6 contains concluding remarks, whereas proofs are presented in the appendix. Some technical details and additional simulation results are deferred to a supplement.

2 The model and test statistic

Let $(Y_{t},\bm{X}_{t})_{t\in\mathbb{Z}}$ be a weakly dependent stochastic process in $\mathbb{R}\times\mathbb{R}^{d}$ following the regression model

[TABLE]

The covariate $\bm{X}_{t}$ may include finitely many lagged values of $Y_{t}$, for instance $\bm{X}_{t}=(Y_{t-1},\dots,Y_{t-d})$ such that the model includes nonparametric autoregression. The unobservable innovations $(U_{t})_{t\in\mathbb{Z}}$ are assumed to fulfill $E[U_{t}|\mathcal{F}^{t}]=0$ almost surely for the sigma-field $\mathcal{F}^{t}=\sigma(U_{j-1},\bm{X}_{j}:j\leq t)$. Our assumptions on the innovations are very weak; in particular heteroscedastic models will be covered. Assuming $(Y_{1},\bm{X}_{1}),\dots,(Y_{n},\bm{X}_{n})$ have been observed, our aim is to test the null hypothesis

[TABLE]

for the conditional mean function $E[Y_{t}|\bm{X}_{t}=\bm{x}]=m_{t}(\bm{x})$, $t\in\mathbb{Z}$, and some not specified function $m:\mathbb{R}^{d}\to\mathbb{R}$ not depending on the time of observation $t$. To test $H_{0}$, we define the sequential marked empirical process of residuals as

[TABLE]

for $s\in[0,1]$ and $\bm{z}\in\mathbb{R}^{d}$, where $\omega_{n}(\cdot)=I\{\cdot\in\bm{J}_{n}\}$ with $\bm{J}_{n}$ from assumption (J) below. Throughout $\bm{x}\leq\bm{y}$ is short for $x_{j}\leq y_{j},\ \forall\ j=1,\dots,d$, and we use the notations $\left\lfloor x\right\rfloor=\max\{k\in\mathbb{Z}:k\leq x\}$ for $x\in\mathbb{R}$ and $\bm{x}\wedge\bm{y}=(\min\{x_{1},y_{1}\},\dots,\min\{x_{d},y_{d}\})$ as well as $\int_{(-\bm{\infty},\bm{x}]}g(\bm{u})d\bm{u}=\int_{-\infty}^{x_{d}}\dots\int_{-\infty}^{x_{1}}g(u_{1},\dots,u_{d})du_{1}\dots du_{d}$; further $I\{\cdot\}$ denotes the indicator function.

The regression function $m$ is estimated by the Nadaraya-Watson estimator $\hat{m}_{n}$, where

[TABLE]

with kernel function $K$ and bandwidth $h_{n}$ as considered in the assumptions below.

The proposed test is a modification of the CUSUM test in Su and Xiao (2008). They consider the process

[TABLE]

where $w:\mathbb{R}^{d}\to\mathbb{R}$ is a weighting function and $\hat{f}_{n}$ is the kernel density estimator. While the factor $\hat{f}_{n}$ has technical reasons as small random values in the denominator of $\hat{m}_{n}$ can be avoided, the weighting function $w$ plays a crucial role for the power of their test (see remarks to Theorem 3.2 in Su and Xiao (2008)). Depending on the alternative, $w$ needs to be chosen appropriately, while their test (in contrast to the one based on the sequential marked process) is not generally consistent.

Under the null hypothesis $H_{0}$ we formulate the following assumptions in order to derive the limiting distribution of $\hat{T}_{n}$ and corresponding test statistics in the next section.

1. (G)

Let $(Y_{t},\bm{X}_{t})_{t\in\mathbb{Z}}$ be strictly stationary and $\alpha$-mixing with mixing coefficient $\alpha(\cdot)$ such that $\alpha(t)=O(a^{-t})$ for some $a\in(1,\infty)$. 2. (U)

For some $\gamma>0$ and some even $Q>(d+1)(2+\gamma)$, and $\mathcal{F}^{t}=\sigma(U_{j-1},\bm{X}_{j}:j\leq t)$, let $E[U_{t}|\mathcal{F}^{t}]=0$, $E[U_{t}^{2}|\bm{X}_{t}]=\sigma^{2}(\bm{X}_{t})$ and $E[|U_{t}|^{Q\frac{2+\gamma}{2}}|\bm{X}_{t}]\leq c(\bm{X}_{t})^{Q}$ a.s. for all $t\in\mathbb{Z}$, for some functions $c,\sigma^{2}:\mathbb{R}^{d}\to\mathbb{R}$ with $\int\bar{c}(\bm{u})dF(\bm{u})\leq M$ for some $M<\infty$ and $\bar{c}(\bm{u})=\max\left\{\sigma^{2}(\bm{u}),c(\bm{u})^{2},\dots,c(\bm{u})^{Q}\right\}$. 3. (M)

For some $b>2$ let $E[|Y_{1}|^{b}]<\infty$ and let $\bm{X}_{1}$ be absolutely continuous with density function $f:\mathbb{R}^{d}\to\mathbb{R}$ that satisfies $\sup_{\bm{x}\in\mathbb{R}^{d}}E[|Y_{1}|^{b}|\bm{X}_{0}=\bm{x}]f(\bm{x})<\infty$ and $\sup_{\bm{x}\in\mathbb{R}^{d}}f(\bm{x})<\infty$. Let there exist some $j^{*}<\infty$ such that $\sup_{\bm{x}_{1},\bm{x}_{j}}E[|Y_{1}Y_{j}||\bm{X}_{1}=\bm{x}_{1},\bm{X}_{j}=\bm{x}_{j}]f_{1j}(\bm{x}_{1},\bm{x}_{j})<\infty$ for all $j\geq j^{*}$, where $f_{1j}$ is the density function of $(\bm{X}_{1},\bm{X}_{j})$. 4. (J)

Let $(c_{n})_{n\in\mathbb{N}}$ be a positive sequence of real valued numbers satisfying $c_{n}\to\infty$ and $c_{n}=O((\log{n})^{1/d})$ and let $\bm{J}_{n}=[-c_{n},c_{n}]^{d}$. 5. (F1)

For some $C<\infty$ and $c_{n}$ from assumption (J) let $\bm{I}_{n}=[-c_{n}-Ch_{n},c_{n}+Ch_{n}]^{d}$ and let $\delta_{n}^{-1}=\inf_{\bm{x}\in\bm{J}_{n}}f(\bm{x})>0$ for all $n\in\mathbb{N}$. Further, let for some $r,l\in\mathbb{N}$ and for all $n\in\mathbb{N}$

[TABLE]

where $|\bm{i}|=\sum_{j=1}^{d}i_{j}$ and $D^{\bm{i}}=\frac{\partial^{|\bm{i}|}}{\partial x_{1}^{i_{1}}\dots\partial x_{d}^{i_{d}}}$ for $\bm{i}=(i_{1},\dots,i_{d})\in\mathbb{N}_{0}^{d}$. 6. (F2)

For $q_{n}$ from assumption (F1), $c_{n}$ from assumption (J) and $C$ from assumption (K), let for all $\bm{k}\in\mathbb{N}_{0}^{d}$ with $|\bm{k}|=2$, $\sup_{\bm{x}\in[-c_{n}-2h_{n}C,c_{n}+2h_{n}C]^{d}}\left|D^{\bm{k}}m(\bm{x})\right|=O(q_{n})$. 7. (K)

Let $K:\mathbb{R}^{d}\to\mathbb{R}$ be symmetric in each component, $l+1$ times differentiable with $\int_{\mathbb{R}^{d}}K(\bm{z})d\bm{z}=1$ and compact support $[-C,C]^{d}$. Additionally, let $r\geq 2$ and $\int_{\mathbb{R}^{d}}K(\bm{z})\bm{z}^{\bm{k}}d\bm{z}=0$ for all $\bm{k}\in\mathbb{N}_{0}^{d}$ with $1\leq|\bm{k}|\leq r-1$, where $\bm{z}^{\bm{k}}=z_{1}^{k_{1}}\cdots z_{d}^{k_{d}}$. For all $L\in\{K\}\cup\{D^{\bm{k}}K:\bm{k}\in\mathbb{N}_{0}^{d}\text{ with }1\leq|\bm{k}|\leq l+1\}$ let $|L(\bm{u})|<\infty$ for all $\bm{u}\in\mathbb{R}^{d}$ and $|L(\bm{u})-L(\bm{u^{\prime}})|\leq\Lambda\|\bm{u}-\bm{u^{\prime}}\|$ for some $\Lambda<\infty$ and for all $\bm{u},\bm{u^{\prime}}\in\mathbb{R}^{d}$. (Here, $r,l$ and $C$ are from assumption (F1).) 8. (B1)

For $\delta_{n},p_{n},q_{n}$ and $r,l$ from assumption (F1) let

[TABLE]

and for some $\eta\in(0,1)$ let

[TABLE] 9. (B2)

For $l,p_{n},q_{n},\delta_{n}$ from assumption (F1) and $\eta$ from assumption (B1), let $h_{n}$ satisfy the following conditions

[TABLE]

Remark 2.1**.**

Under aforementioned assumptions, consistency properties hold for $\hat{m}_{n}$ uniformly on $\bm{J}_{n}$ from assumption (J) which will be shown in section A.1 of the appendix. The key tool here is an application of Theorem 2 in Hansen (2008). Assumption (G) implies polynomial mixing rates of the underlying process needed in Hansen (2008). Moreover, together with the first bandwidth condition in (B2) the bandwidth constraints in Hansen (2008) are also fulfilled. Assumptions (M) and parts of (K) are reproduced from aforementioned paper.

*In order to satisfy the first bandwidth condition in (B2), a necessary condition on the smoothness of $f$ and $m$ then is $l+\eta>d$, meaning that for higher dimensional covariate $\bm{X}_{t}$, the existence of higher order partial derivatives of $f$ and $m$ is needed. In order to satisfy both the first and third bandwidth condition in (B2) at the same time, the order of the kernel needs to be large, in particular $r>\frac{d}{2}\frac{l+\eta}{l+\eta-d}$. The second bandwidth condition in (B2) is implied by the first one, if the bandwidth $h_{n}$ has a polynomial rate of decay in $n$ (or slower), meaning if there exists a $k\in(0,\infty)$ such that $h_{n}=O(n^{-k})$. Note that $k<\frac{1}{d}-\frac{1}{l+\eta}$ is necessary then. *

3 Asymptotic results

To derive the asymptotic distribution of test statistics built from the sequential marked empirical process $\hat{T}_{n}$ defined in (2.1), we apply the following expansion, which uses $Y_{i}=m(\bm{X}_{i})+U_{i}$ for all $i=1,\dots,n$ under the null hypothesis,

[TABLE]

with

[TABLE]

Lemma A.3 in the appendix shows that $A_{n2}(s,\bm{z})=T_{n}(s,\bm{z})+o_{P}(1)$ uniformly in $s\in[0,1]$ and $\bm{z}\in\mathbb{R}^{d}$ with the process

[TABLE]

Further, Lemma A.2 in the appendix shows that

[TABLE]

holds uniformly in $s\in[0,1]$ and $\bm{z}\in\mathbb{R}^{d}$. Inserting the definition of $\hat{m}_{n}$ from (2.2) one obtains one term of the form

[TABLE]

which is negligible by Lemma A.4 and one term of the form

[TABLE]

which can further be expanded applying Lemmata A.5 and A.3 such that one obtains

[TABLE]

uniformly in $s\in[0,1]$ and $\bm{z}\in\mathbb{R}^{d}$. From this the expansion given in the first part of Theorem 3.1 below follows. In the second part of the theorem weak convergence of $T_{n}$ from (3.3) is stated.

Theorem 3.1**.**

(i)* Suppose that (G), (U), (M), (J), (F1), (F2), (K), (B1) and (B2) are satisfied. Then under $H_{0}$*

[TABLE]

holds uniformly in $s\in[0,1]$ and $\bm{z}\in\mathbb{R}^{d}$.

(ii)* Suppose that the assumptions (G) and (U) are satisfied. Then under $H_{0}$ the process $T_{n}$ converges weakly in $\ell^{\infty}([0,1]\times\mathbb{R}^{d})$ to a centered Gaussian process $G$ with*

[TABLE]

and $\Sigma:\mathbb{R}^{d}\to\mathbb{R},\bm{x}\mapsto\int_{(-\bm{\infty},\bm{x}]}\sigma^{2}(\bm{u})f(\bm{u})d\bm{u}$.

The proof of the first part follows from the considerations above applying Lemmata A.2–A.5 in the appendix, while the proof of the second part is given in section A.2 of the appendix. The proof of the second part in particular makes use of a recent result on weak convergence of sequential empirical processes indexed in function classes that can be applied for strongly mixing sequences, see Mohr (2018b). Note that Koul and Stute (1999) show a weak convergence result applicable to the non-sequential process $\{T_{n}(1,z):z\in\mathbb{R}\}$ under less restrictive assumptions on the dependence structure and moments (see Lemma 3.1 in aforementioned reference). From Theorem 3.1 and the continuous mapping theorem one directly obtains the limit distribution of $\hat{T}_{n}$.

Corollary 3.2**.**

Suppose that the assumptions of Theorem 3.1(i) are satisfied. Then under $H_{0}$ the process $\hat{T}_{n}$ converges weakly in $\ell^{\infty}([0,1]\times\mathbb{R}^{d})$ to a centered Gaussian process $G_{0}$ with

[TABLE]

and $\Sigma$ as in Theorem 3.1(ii).

Continuous functionals of the process $\hat{T}_{n}$ can be used as test statistics for $H_{0}$. We consider the following Kolmogorov-Smirnov and Cramér-von Mises type statistics and combinations of both,

[TABLE]

where $v:\mathbb{R}^{d}\to\mathbb{R}$ is some integrable weighting function. Applying Corollary 3.2 and the continuous mapping theorem gives convergence in distribution of those test statistics. One can obtain distribution-free tests in the case of dimension $d=1$ as follows. Denote by $\{K_{0}(s,t):s\in[0,1],t\in\mathbb{R}\}$ a Kiefer-Müller process, i.e. a centered Gaussian process with covariance function $\operatorname{Cov}(K_{0}(s_{1},t_{1}),K_{0}(s_{2},t_{2}))=(s_{1}\wedge s_{2}-s_{1}s_{2})(t_{1}\wedge t_{2})$. Then $K_{0}(\cdot,\Sigma(\cdot))$ has the same distribution as $G_{0}(\cdot,\cdot)$. Let further $\sigma(\cdot)$ be continuous and consider the consistent estimator $\hat{c}_{n}=n^{-1}\sum_{i=1}^{n}(Y_{i}-\hat{m}_{n}(X_{i}))^{2}\omega_{n}(X_{i})$ for $c=\int\sigma^{2}(u)f(u)du$. Applying a scaling property of the process $K_{0}$ in its second component and substitution in the integrals it is easy to derive convergence in distribution as follows,

[TABLE]

For the latter two tests however the unknown weight function $v=\sigma^{2}f$ needs to be chosen to obtain the limit as stated above. To obtain feasible asymptotically distribution-free tests, $T_{n3}$ and $T_{n4}$ should be replaced by

[TABLE]

applying a nonparametric estimator for the variance function such as

[TABLE]

To conclude the section we will have a closer look at the alternative of one change point. For simplicity reasons we will only consider the test based on $T_{n1}$. To model the alternative we assume a triangular array

[TABLE]

and validity of the alternative of one change point, i.e.

[TABLE]

for some not further specified functions $m_{(1)}\not\equiv m_{(2)}$. Let $f_{n,t}$ denote the density of $\bm{X}_{n,t}$ and assume that for all $s\in(0,1]$ there exists a function $\bar{f}^{(s)}:\mathbb{R}^{d}\to\mathbb{R}$ such that

[TABLE]

Under some regularity conditions it can be shown by applying Kristensen’s (2009) results that

[TABLE]

where $\bar{m}_{n}(\bm{x})=\sum_{i=1}^{n}f_{n,i}(\bm{x})m_{n,i}(\bm{x})/\sum_{i=1}^{n}f_{n,i}(\bm{x})$ converges to the mixture

[TABLE]

of the regression functions before and after the change. Now for fixed $\bm{z}\in\mathbb{R}^{d}$ and $s\in(0,1)$ with $s\leq s_{0}$, it holds that

[TABLE]

where

[TABLE]

As under $H_{1}$ this integral is non-zero for $s=s_{0}$ and some $\bm{z}$, convergence of $T_{n1}$ to infinity in probability and thus consistency of the test can be deduced.

Remark 3.3**.**

Consider the non-marked CUSUM process $\hat{T}_{n}(s,\bm{\infty})$ which is analogous to Su and Xiao’s (2008) procedure. Considerations as above for the fixed alternative $H_{1}$ of one change point in $\left\lfloor ns_{0}\right\rfloor$ leads for $s\leq s_{0}$ to

[TABLE]

where the last equality holds in case of a stationary covariate process. The integral can be zero even if $m_{(1)}\neq m_{(2)}$. Then tests based on the CUSUM process will not be consistent, while tests based on the marked CUSUM process are. We will consider some examples in section 5.

4 A bootstrap procedure and the case of non-stationary variances

As alternative to the asymptotic test considered in section 3, in this section we will suggest a wild bootstrap approach. This resampling procedure can in particular be applied in the case of multivariate covariates, where the critical values for the asymptotic tests based on Corollary 3.2 have to be estimated. Moreover, the bootstrap approach can be applied to obtain a test that detects changes in the conditional mean function, even when the conditional variance function is not stable. As desired, the test does not react sensitive to the unstable variance. In contrast to the bootstrap approach, the limiting distribution from section 3 cannot be applied in the case of changes in the variance.

We consider the model

[TABLE]

with $E[U_{n,t}|\mathcal{F}_{n}^{t}]=0$ and $E[U_{n,t}^{2}|\bm{X}_{n,t}]=\sigma^{2}_{n,t}(\bm{X}_{n,t})$ a.s. for some functions $\sigma^{2}_{n,t}:\mathbb{R}^{d}\to\mathbb{R}$ and $\mathcal{F}_{n}^{t}:=\sigma(U_{n,j-1},\bm{X}_{n,j}:j\leq t)$. We assume $\bm{X}_{n,t}$ to be absolutely continuous with density function $f_{n,t}$. The model considered in section 2 and the first part of section 3 is the special case where $f_{n,t}(\cdot)=f(\cdot)$ and $\sigma^{2}_{n,t}(\cdot)=\sigma^{2}(\cdot)$ for all $t=1,\dots,n$ and for some $f,\sigma^{2}:\mathbb{R}^{d}\to\mathbb{R}$ not depending on $t$ and $n$. Both models allow for heteroscedasticity, but the more general model also allows for possible changes in $\sigma^{2}_{n,t}$, which should not effect the rejection probability of the test for

[TABLE]

(for some $m$ not depending on $t$ and $n$). We again consider the procedure

[TABLE]

with residuals $\hat{U}_{n,i}=Y_{n,i}-\hat{m}_{n}(\bm{X}_{n,i})$. Here $\hat{m}_{n}$ is defined as in (2.2), but replacing $(\bm{X}_{j},Y_{j})$ by $(\bm{X}_{n,j},Y_{n,j})$, $j=1,\dots,n$.

First define the wild bootstrap innovations as $U_{n,t}^{*}=\hat{U}_{n,t}\eta_{t}$, where $\{\eta_{t}\}$ are i.i.d. random variables, independent of the original sample with $E[\eta_{0}]=0$, $E[\eta_{0}^{2}]=1$ and $E[\eta_{0}^{4}]<\infty$. Then the bootstrap data fulfilling the null hypothesis are generated by

[TABLE]

Note that if the original data follow an autoregression model, say $d=1$ and $X_{n,t}=Y_{n,t-1}$, by the above choice the resulting bootstrap data does not follow the same structure. As was pointed out by Kreiß and Lahiri (2012) this bootstrap data generation is still a reasonable choice in particular if the dependence structure of the underlying process does not show up in the asymptotic distribution. Another possibility might be a dependent wild bootstrap as suggested in Shao (2010).

The bootstrap residuals are defined as $\hat{U}^{*}_{n,t}=Y^{*}_{n,t}-\hat{m}^{*}_{n}(\bm{X}_{n,t})$, where $\hat{m}^{*}_{n}$ is defined as $\hat{m}_{n}$ in (2.2), but replacing $(\bm{X}_{j},Y_{j})$ by $(\bm{X}_{n,j},Y_{n,j}^{*})$, $j=1,\dots,n$. The bootstrap process is defined as

[TABLE]

Bootstrap versions $T_{n\ell}^{*}$, $\ell=1,\dots,4$, are defined analogous to the test statistics $T_{n\ell}$, $\ell=1,\dots,4$, but based on $\hat{T}_{n}^{*}$ instead of $\hat{T}_{n}$. Then $H_{0}$ is rejected if $T_{n\ell}$ is larger than the ($1-\alpha$)-quantile of the conditional distribution of $T_{n\ell}^{*}$, given the original data.

To motivate that we obtain a valid procedure (which holds the level asymptotically and is consistent) even in the case of changing variances, we will consider the limiting process $G_{0}$ of the original process $\hat{T}_{n}$ and the conditional limiting process $G_{0}^{*}$ of the bootstrap version $\hat{T}^{*}_{n}$ in subsections 4.1 and 4.2 below. We will see that the processes $G_{0}$ and $G_{0}^{*}$ coincide under the null hypothesis. Note that some steps of the derivation are explained heuristically, whereas rigorously deriving the weak convergence would require a limit theorem for sequential empirical processes indexed in function classes for weakly dependent non-stationary data. Such a result is, to the best of our knowledge, not yet available in the literature and thus a rigorous proof is beyond the scope of the paper (see Mohr (2018b) for a related limit theorem that requires stationarity).

4.1 Asymptotics for non-homogeneous variances

Heuristically under $H_{0}$ one can proceed as in the proof of the first part of Theorem 3.1 in the beginning of section 3. Again one has the expansion $\hat{T}_{n}(s,\bm{z})=A_{n2}(s,\bm{z})+A_{n1}(s,\bm{z})$, however, similar to Lemma A.2 in the appendix one will now obtain

[TABLE]

uniformly in $s\in[0,1]$ and $\bm{z}\in\mathbb{R}^{d}$ under suitable regularity conditions and under the assumption that the limit $\bar{f}^{(s)}$ as in (3.5) exists. Inserting the definition of $\hat{m}_{n}$ analogously to Lemmata A.4 and A.5 this will result in one negligible term and one term of the form

[TABLE]

Further, Lemma A.3 will stay valid in analogous form. Thus, one obtains the exansion

[TABLE]

with the process

[TABLE]

where $\bar{g}^{(t)}:=\bar{f}^{(t)}/\bar{f}^{(1)}$. Now assume that the limit $\bar{h}^{(s)}(\cdot):=\lim_{n\to\infty}n^{-1}\sum_{i=1}^{\left\lfloor ns\right\rfloor}\sigma_{n,i}^{2}(\cdot)f_{n,i}(\cdot)$ exists for all $s\in(0,1]$ and that the process $\Gamma_{n}$ converges weakly to a centered Gaussian process $\Gamma$ (a proof would require a weak convergence result for sequential empirical processes indexed in general function classes and with a weakly dependent and non-stationary underlying triangular array process). The limiting covariance is

[TABLE]

Then with the continuous mapping theorem the weak convergence of $\hat{T}_{n}$ to a centered Gaussian process $\{G_{0}(s,\bm{z}):s\in[0,1],\bm{z}\in\mathbb{R}^{d}\}$ follows with covariances

[TABLE]

Note that this is consistent with the stationary case as then $\bar{h}^{(s)}(\cdot)=s\sigma^{2}(\cdot)f(\cdot)$ and $\bar{g}^{(s)}(\cdot)=s$ and the same covariance function as in Corollary 3.2 is obtained. The convergence of the test statistics $T_{n\ell}$, $\ell=1,\dots,4$, in distribution follows again from the continuous mapping theorem.

Under the change point alternative $H_{1}$ from (3.4) with $m_{(1)}\not\equiv m_{(2)}$, analogous to the considerations in section 3 it holds that the test statistic $T_{n1}$ converges to infinity in probability.

4.2 Derivations for the bootstrap process

Concerning the weak convergence of the bootstrap process $\hat{T}_{n}^{*}$, conditionally on the sample, we have again a look at the expansion in the beginning of section 3 for the derivation of the first part of the proof of Theorem 3.1. In what follows let $P^{*}$ denote the conditional probability and $E^{*}$ the conditional expectation, given the observations. Further let $Z_{n}=o_{P^{*}}(1)$ be short for $P^{*}(|Z_{n}|>\epsilon)=o_{P}(1)$ for all $\epsilon>0$. Here we obtain

[TABLE]

with

[TABLE]

and (similar to Lemma A.2 in the appendix)

[TABLE]

with $\bar{f}^{(s)}$ as in (3.5). Inserting the definition of $\hat{m}_{n}^{*}$ this leads to a term (similar to Lemma A.4) of the form

[TABLE]

which is negligible, and a term (similar to Lemma A.5) of the form

[TABLE]

Thus one obtains (under suitable regularity conditions) the expansion

[TABLE]

where

[TABLE]

and $\bar{g}^{(t)}$ is defined as in section 4.1. In what follows we will assume that the process $\Gamma^{*}_{n}$, conditionally on the sample, converges weakly to a centered Gaussian process, in probability. Then, by the continuous mapping theorem, $\hat{T}^{*}_{n}$, conditionally converges weakly to a centered Gaussian process, say $G_{0}^{*}$. We will calculate the asymptotic variances in order to show that under $H_{0}$ those coincide with the covariances of $G_{0}$ as in section 4.1. First note that $E^{*}[U_{n,i}^{*}U_{n,j}^{*}]=\hat{U}_{n,i}^{2}I\{i=j\}$ almost surely. Under $H_{0}$ it holds that $\hat{U}_{n,t}=m(\bm{X}_{n,t})-\hat{m}_{n}(\bm{X}_{n,t})+U_{n,t}$ and $\hat{m}_{n}$ consistently estimates $m$, and thus

[TABLE]

under $H_{0}$, where $\Gamma$ is the limiting distribution of $\Gamma_{n}$ in section 4.1. Thus, under $H_{0}$, $\hat{T}^{*}_{n}$ indeed (presumably) converges weakly to $G_{0}$ in probability, and thus the test statistic $T_{n\ell}^{*}$ converges conditionally in distribution, to the same limits as $T_{n\ell}$ (respectively for $\ell=1,\dots,4$).

Under the alternative $H_{1}$ as in (3.4), $\hat{U}_{n,i}=m_{n,i}(\bm{X}_{n,i})-\hat{m}_{n}(\bm{X}_{n,i})+U_{n,i}$ and thus it holds that

[TABLE]

for fixed $s_{1},s_{2},t_{1},t_{2}\in[0,1]$ and $\bm{z}_{1},\bm{z}_{2}\in\mathbb{R}^{d}$. The first term again converges in probability to $E\left[\Gamma(s_{1},t_{1},\bm{z}_{1})\Gamma(s_{2},t_{2},\bm{z}_{2})\right]$. It can further be shown that

[TABLE]

converges to zero in probability. However,

[TABLE]

with the same $\bar{m}_{n}$ as in (3.6), which converges to

[TABLE]

(see (3.7)). Thus, it can be shown that

[TABLE]

It can be seen that these terms do not vanish but converge to some limit in probability. Thus the limiting distribution $G_{0}^{*}$ under $H_{1}$ is not equal to $G_{0}$ and in particular depends on the changepoint $s_{0}$. As seen before under $H_{1}$ the original test statistic $T_{n1}$ converges in probability to infinity. On the other hand, the bootstrap test statistic $T_{n1}^{*}$ conditionally converges in distribution to some non-degenerated limit, in probability. Thus the bootstrap test is consistent.

5 Finite sample properties

A small Monte Carlo study is conducted in order to compare the results for $T_{n1}$ and $T_{n2}$ from section 3 with those of the traditional CUSUM versions denoted by $KS:=\sup_{s\in[0,1]}|\hat{T}_{n}(s,\infty)|$ and $CM:=\int|\hat{T}_{n}(s,\infty)|^{2}ds$. Note that the results for $\tilde{T}_{n3}$ and $\tilde{T}_{n4}$ are similar and omitted for reasons of brevity. Asymptotic tests are applied to data satisfying models 1 and 2, while the bootstrap versions are applied to model 3 explained below. Also note that simulation results for a multidimensional autoregression model can additionally be found in the supplement. All simulations are carried out with a level of $5\%$, $500$ replications and $200$ bootstrap replications and for sample sizes $n\in\{100,300,500\}$. For the nonparametric estimators we use a fourth order Epanechnikov kernel and the bandwidth is chosen by the cross validation method. For simplicity we set $\omega_{n}\equiv 1$. The data is simulated from the following models.

[TABLE]

where $X_{i}$ is an exogenous variable following the AR(1) model $X_{t}=0.4X_{t-1}+\xi_{t}$ with $\xi_{i}$ being i.i.d. $\sim\mathcal{N}(0,1)$ and $\Delta_{0}\in\{0,0.5,1,1.5,1.5,2,2.5,3,3.5,4\}$.

[TABLE]

with $\Delta_{0}\in\{0,0.2,0.4,0.6,0.8,1,1.2,1.4,1.6,1.8\}$. Consider the homoscedastic case, where $\sigma^{2}(x)=1$ and the heteroscedastic case, where $\sigma^{2}(x)=1+0.1x^{2}$.

[TABLE]

with $\Delta_{0}\in\{0,1.3\}$ and $t_{0}\in\{0.25,0.5,0.75\}$.

Model 1 is a regression model with autoregressive covariables. In model 2 we consider both a homoscedastic and heteroscedastic autoregression model, while model 3 is a heteroscedastic autoregression with non-homogeneous variances. All models fulfill $H_{0}$ for $\Delta_{0}=0$ and $H_{1}$ for $\Delta_{0}\neq 0$ with a change in regression function occurring in $\left\lfloor n/2\right\rfloor$. Further, note that models 1 and 2 fulfill the stationarity and mixing assumptions under $H_{0}$, while model 3 is not stationary as one change occurs in the conditional variance function under both $H_{0}$ and $H_{1}$. Hence, for model 3 we apply the bootstrap test from section 4.

Figure 1, 2 and 3 are visualizations of the performance of $T_{n1}$ and $T_{n2}$, as well as $KS$ and $CM$ in model 1 and 2. Under the null the rejection frequencies for all tests are near the nominal level. For model 1 the CUSUM tests are not consistent against $H_{1}$, while the tests based on the marked process are. In model 2 the rejection frequencies of all tests increase with increasing break size. Note however that the increase is much faster for $T_{n1}$ and $T_{n2}$ than for the CUSUM tests. Also note that the influence of the conditional variance is rather small resulting in a similar performance in both the homoscedastic and heteroscedastic case. Table 1 shows the rejection frequencies of the bootstrap procedure using $T^{*}_{n1}$ and $T^{*}_{n2}$, as well as the bootstrap version of the CUSUM tests $KS$ and $CM$ under both the null and the alternative hypothesis. The level simulations show that all tests perform reasonably well under $H_{0}$, approximately holding the level indicating that the bootstrap test is – as desired – not sensitive to changes in the conditional variance function. Furthermore, it can be seen that for all models and all tests the rejection frequency under $H_{1}$ exceeds the level, indicating that the change point is detected. With increasing sample size, the number of rejections increases rapidly for $T_{n1}^{*}$ and $T_{n2}^{*}$, while it stays approximately constant for the bootstrap versions of $KS$ and $CM$. This is presumably due to the fact that the test statistics based on $\hat{T}_{n}(s,\infty)$ estimate some integral that might be small under $H_{1}$. As was pointed out in subsection 4.2, this integral not vanishing is essential for the consistency property for the bootstrap tests.

Finally, we apply the asymptotic test based on $T_{n1}$ to $36$ measurements of the annual flow volume of the small Czech river Ráztoka recorded between 1954 and 1989. It was considered by Hušková and Antoch (2003). We set $X_{t}$ as the annual rainfall and $Y_{t}$ as the annual flow volume. The asymptotic test clearly rejects $H_{0}$ with a $p$-value of $0.0006$. The possible change point is estimated by $\hat{s}_{n}$ from section 6 and suggests a change in 1979. Note that this is consistent with the literature. As was pointed out by Hušková and Antoch (2003) deforestation had started around that time, which is a possible explanation. Figure 4 shows on the left-hand side the scatterplot $X_{t}$ against $Y_{t}$ using dots for the observations after the estimated change and crosses for the observations before the estimated change. On the right-hand side the figure shows the cumulative sum, $\sup_{z\in\mathbb{R}}|\hat{T}_{n}(\cdot,z)|$, as well as the critical value (red horizontal line) and the estimated change (green vertical line).

6 Concluding remarks

We suggested a new test for structural breaks in the regression function in nonparametric time series (auto-)regression. Our approach combines CUSUM statistics with the marked empirical process approach from goodness-of-fit testing. The considered model is very general and does not require independent innovations, nor homoscedasticity. We show favorable asymptotic properties and demonstrate that the new testing procedures are consistent against fixed alternatives, while the traditional CUSUM tests are not. An estimator for the change point is given by $\hat{s}_{n}:=\arg\max_{s\in[0,1]}\sup_{\bm{z}\in\mathbb{R}^{d}}|\hat{T}_{n}(s,\bm{z})|$. Asymptotic properties of this estimator will be considered in future research.

Moreover we have suggested a bootstrap version that can also be applied to detect changes in the regression function in the presence of changing variance functions. In a forthcoming paper we will consider testing for changes in the variance function.

Appendix A Proofs and derivations

In subsection A.1 we give some auxiliary results for the proof of Theorem 3.1. The proof of the first part of the theorem was given in the main text, while the proof of the second part can be found in subsection A.2. Some lemmata are proved in subsection A.3 while some details are deferred to the supplement. Detailed proofs can also be found in Mohr (2018a).

A.1 Auxiliary results

The following assumptions are formulated for the first lemma that gives uniform rates of convergence for the regression estimator $\hat{m}_{n}$ from (2.2) and its derivatives. They hold under the assumptions of Theorem 3.1.

1. (P)

Let $(Y_{t},\bm{X}_{t})_{t\in\mathbb{Z}}$ be a strictly stationary and strongly mixing process with mixing coefficient $\alpha(\cdot)$. For some $b>2$ let $\alpha(t)=O(t^{-\beta})$ for $t\to\infty$ with some $\beta>(1+(b-1)\left(1+d\right))/(b-2)$. 2. (B3)

With $b$ and $\beta$ from assumption (P) let $(\log n)/(n^{\theta}h_{n}^{d})=o(1)$ for $\theta=(\beta-1-d-(1+\beta)/(b-1))/(\beta+3-d-(1+\beta)/(b-1))$.

Lemma A.1**.**

Under the assumptions (P), (M), (J), (F1), (K), (B1) and (B3) the following rates of convergence can be obtained for the Nadaraya-Watson estimator $\hat{m}_{n}$,

1. (a)

$\sup\limits_{\bm{x}\in\bm{J}_{n}}\left|\hat{m}_{n}(\bm{x})-m(\bm{x})\right|=O_{P}\left(\left(\sqrt{\frac{\log{(n)}}{nh_{n}^{d}}}+h_{n}^{r}p_{n}\right)q_{n}\delta_{n}\right)$, 2. (b)

$\sup\limits_{\bm{x}\in\bm{J}_{n}}\left|D^{\bm{k}}\left(\hat{m}_{n}(\bm{x})-m(\bm{x})\right)\right|=O_{P}\left(\left(\sqrt{\frac{\log{(n)}}{nh_{n}^{d+2|\bm{k}|}}}+h_{n}^{r}p_{n}\right)p_{n}^{|\bm{k}|}q_{n}\delta_{n}^{|\bm{k}|+1}\right)$* for all $\bm{k}\in\mathbb{N}_{0}^{d}$ with $1\leq|\bm{k}|\leq l+1$,* 3. (c)

$\displaystyle\sup\limits_{\begin{subarray}{c}\bm{x},\bm{y}\in\bm{J}_{n}\\ \bm{x}\neq\bm{y}\end{subarray}}\frac{\left|D^{\bm{k}}\left(\hat{m}_{n}(\bm{x})-m(\bm{x})\right)-D^{\bm{k}}\left(\hat{m}_{n}(\bm{y})-m(\bm{y})\right)\right|}{\|\bm{x}-\bm{y}\|^{\eta}}=o_{P}(1)$* for all $\bm{k}\in\mathbb{N}_{0}^{d}$ with $|\bm{k}|=l$. *

The proof of Lemma A.1 is analogous to the proof of Theorem 8 of Hansen (2008) and omitted for the sake of brevity. The proofs of the following lemmata are given in subsection A.3.

Lemma A.2**.**

Under the assumptions of Theorem 3.1 (i) and under $H_{0}$ we have for $A_{n1}$ from (3.1)

[TABLE]

uniformly in $s\in[0,1]$ and $\bm{z}\in\mathbb{R}^{d}$.

Lemma A.3**.**

Under the assumptions of Theorem 3.1 (i) and under $H_{0}$ we have for $A_{n2}$ from (3.2) and $T_{n}$ from (3.3)

[TABLE]

uniformly in $s\in[0,1]$ and $\bm{z}\in\mathbb{R}^{d}$.

Lemma A.4**.**

Under the assumptions of Theorem 3.1 (i) and under $H_{0}$

[TABLE]

holds uniformly in $\bm{z}\in\mathbb{R}^{d}$.

Lemma A.5**.**

Under the assumptions of Theorem 3.1 (i) and under $H_{0}$

[TABLE]

holds uniformly in $\bm{z}\in\mathbb{R}^{d}$.

A.2 Proof of Theorem 3.1(ii)

For the proof of the second part of Theorem 3.1 we use a recent result on weak convergence of sequential empirical processes indexed in function classes that can be applied for strongly mixing sequences, see Mohr (2018b). It is stated in Lemma A.7 and uses the following notion of bracketing number.

Definition A.6** (Bracketing number).**

Let $\mathcal{X}$ be a measure space, $\mathcal{F}$ some class of functions $\mathcal{X}\to\mathbb{R}$ and $\rho$ some semi norm on $\mathcal{F}$. For all $\varepsilon>0$, let $N=N(\varepsilon)$, be the smallest integer, for which there exist a class of functions $\mathcal{X}\to\mathbb{R}$, denoted by $\mathcal{B}$ and called bounding class and a function class $\mathcal{A}\subset\mathcal{F}$ called approximating class such that $|\mathcal{B}|=|\mathcal{A}|=N$, $\rho(b)<\varepsilon,\ \forall\ b\in\mathcal{B}$ and for all $\varphi\in\mathcal{F}$ there exist $a^{*}\in\mathcal{A}$ and $b^{*}\in\mathcal{B}$ such that $|\varphi-a^{*}|\leq b^{*}$. Then $N(\varepsilon)$ is called the bracketing number and denoted by $\tilde{N}_{[~{}]}(\varepsilon,\mathcal{F},\rho)$.

Note that the usual notion for bracketing number (as in Definition 2.1.6 in van der Vaart and Wellner (1996)) will be referred to as $N_{[~{}]}(\varepsilon,\mathcal{F},\rho)$.

Lemma A.7** (Corollary 2.7 in Mohr (2018b)).**

Let $\{X_{t}:t\in\mathbb{Z}\}$ be a strictly stationary sequence of random variables with values in some measure space $\mathcal{X}$. Let $\mathcal{F}$ be a class of measurable functions $\mathcal{X}\to\mathbb{R}$. Let furthermore the following assumptions hold.

1. (A1)

Let $\{X_{t}:t\in\mathbb{Z}\}$ be strongly mixing, such that $\sum_{t=1}^{\infty}t^{Q-2}\alpha(t)^{\gamma/(2+\gamma)}<\infty$ for some $\gamma>0$ and even $Q>2$. 2. (A2)

For $Q$ and $\gamma$ from assumption (A1) let $\int_{0}^{1}x^{-\gamma/(2+\gamma)}(\tilde{N}_{[~{}]}(x,\mathcal{F},\|\cdot\|_{L_{2}(P)}))^{1/Q}dx<\infty$, where $X_{1}\sim P$. Furthermore, assume that each $\varepsilon>0$ allows for a choice of bounding class $\mathcal{B}$, such that $E\left[|b(X_{1})|^{i(2+\gamma)/2}\right]^{1/2}\leq\varepsilon$ for all $b\in\mathcal{B}$ and for all $i=2,\dots,Q$. 3. (A3)

Let $\mathcal{F}$ possess an envelope function $F$, with $E[|F(X_{1})|^{Q}]<\infty$ and let there exist a constant $L<\infty$, such that $\sup_{\varphi\in\mathcal{F}}E\left[|\varphi(X_{1})|^{Q(2+\gamma)/2}\right]\leq L$.

*Furthermore, let for all $K\in\mathbb{N}$ and all finite collections $\varphi_{k}\in\mathcal{F}$, $s_{k}\in[0,1]$, $k=1,\dots,K$, $\left(G_{n}(s_{k},\varphi_{k})\right)_{k=1,\dots,K}\overset{\mathcal{D}}{\rightarrow}\left(G(s_{k},\varphi_{k})\right)_{k=1,\dots,K}$, where $G_{n}(s,\varphi)=n^{-1/2}\sum_{i=1}^{\left\lfloor ns\right\rfloor}(\varphi(X_{i})-E[\varphi(X_{i})])$ for $s\in[0,1],\varphi\in\mathcal{F}$ and $G=\{G(s,\varphi):s\in[0,1],\varphi\in\mathcal{F}\}$ is a centered Gaussian process.

Then $\left\{G_{n}(s,\varphi):s\in[0,1],\varphi\in\mathcal{F}\right\}$ converges weakly to $G$ in $\ell^{\infty}([0,1]\times\mathcal{F})$.

Proof of Theorem 3.1(ii).

First notice that due to assumption (G) and under the null hypothesis $(U_{t},\bm{X}_{t})_{t\in\mathbb{Z}}$ is a strictly stationary sequence of random variables with values in $\mathbb{R}\times\mathbb{R}^{d}$. Denote by $P$ the common marginal distribution of $(U_{1},\bm{X}_{1})$ and define $\mathcal{F}:=\{(u,\bm{x})\mapsto uI\{\bm{x}\leq\bm{z}\}:\bm{z}\in\mathbb{R}^{d}\}$. The convergence of $T_{n}$ is then implied by the weak convergence of

[TABLE]

in $\ell^{\infty}([0,1]\times\mathcal{F})$. We apply Lemma A.7. Condition (A1) on the mixing coefficient of $(U_{t},\bm{X}_{t})_{t\in\mathbb{Z}}$ is implied by assumption (G) on the mixing coefficient of $(Y_{t},\bm{X}_{t})_{t\in\mathbb{Z}}$ and the null hypothesis as measurable functions maintain mixing properties. To show condition (A2) on the function class $\mathcal{F}$, the choice of approximating functions and bounding functions, as in Definition A.6, will be discussed in more detail. Denote with $\bar{c}$ from assumption (U), $h(\bm{x})=\bar{c}(\bm{x})f(\bm{x})$ and $H(\bm{x})=\int_{(-\bm{\infty},\bm{x}]}h(\bm{t})d\bm{t}$ for $\bm{x}\in\mathbb{R}^{d}$ and for all $i=1,\dots,d$ and $x\in\mathbb{R}$,

[TABLE]

and $H_{i}(x)=\int_{-\infty}^{x}h_{i}(t)dt$. Let $\varepsilon>0$ and choose for all $i=1,\dots,d$ some $N_{i}=N_{i}(\varepsilon)\in\mathbb{N}$ and $-\infty=z_{0,i}<\dots<z_{N_{i},i}=\infty$, such that

[TABLE]

Since $H_{i}$ is continuous and $H_{i}(-\infty)=H(-\bm{\infty})=0$ and $H_{i}(\infty)=H(\bm{\infty})\leq M$ for $M<\infty$ from assumption (U), $N_{i}$ can be chosen to be smaller than $2dM\varepsilon^{-2}$ for all $i=1,\dots,d$. By using cartesian products, a partition of $\mathbb{R}^{d}$ is obtained. For simplicity reasons the following notation will be used. For $\bm{j}=(j_{1},\dots,j_{d})\in\mathbb{N}^{d}$ let $\bm{z}_{\bm{j}}:=(z_{j_{1},1},\dots,z_{j_{d},d})$, and $\bm{j-1}:=(j_{1}-1,\dots,j_{d}-1)\in\mathbb{N}^{d}$. For all $\bm{j}\in\operatorname*{\times}_{i=1}^{d}\{1,\dots,N_{i}\}$ define approximating functions

[TABLE]

and bounding functions

[TABLE]

Notice that $a_{\bm{j}}\in\mathcal{F}$ while $b_{\bm{j}}\notin\mathcal{F}$ for all $\bm{j}\in\operatorname*{\times}_{i=1}^{d}\{1,\dots,N_{i}\}$. For each $\bm{z}\in\mathbb{R}^{d}$ there exists a $\bm{j}\in\operatorname*{\times}_{i=1}^{d}\{1,\dots,N_{i}\}$ such that $\bm{z}\in(\bm{z}_{\bm{j-1}},\bm{z}_{\bm{j}}]$. Therefore for each $\varphi\in\mathcal{F}$ there exists a $\bm{j}\in\operatorname*{\times}_{i=1}^{d}\{1,\dots,N_{i}\}$ such that $|\varphi-a_{\bm{j}}|\leq b_{\bm{j}}$. Further for $\bm{j}\in\operatorname*{\times}_{i=1}^{d}\{1,\dots,N_{i}\}$ it holds that

[TABLE]

due to (A.1). Furthermore for all $i=2,\dots,Q$ by Jensen’s inequality and (U), it holds that

[TABLE]

and thus

[TABLE]

Since $N_{i}=O(\varepsilon^{-2})$ for all $i=1,\dots,d$, it holds that $\tilde{N}_{[~{}]}(\varepsilon,\mathcal{F},\|\cdot\|_{L_{2}(P)})=O\left(\varepsilon^{-2d}\right)$. As $Q>d(2+\gamma)$ holds, assumption (A2) from Lemma A.7 is therefore satisfied. Assumption (A3) is also satisfied as $\bar{F}:\mathbb{R}\times\mathbb{R}^{d}\to\mathbb{R},\ (u,\bm{x})\mapsto u$ is an envelope function of $\mathcal{F}$ such that

[TABLE]

and additionally, it holds that

[TABLE]

What is left to show, is the convergence of all finite dimensional distributions of $T_{n}$. To this end we will apply Cramér-Wold’s device. Let $\lambda_{1},\dots,\lambda_{K}\in\mathbb{R}\setminus\{0\}$ and consider

[TABLE]

where $\xi_{n,i}:=\frac{1}{\sqrt{n}}U_{i}\sum_{j=1}^{K}\lambda_{j}I\{\bm{X}_{i}\leq\bm{z}_{j}\}I\left\{\tfrac{i}{n}\leq s_{j}\right\}$. Now Corollary 1 in Rio (1995) can be applied, which is a central limit theorem for strongly mixing triangular arrays. Following the notations in Rio (1995) define $V_{n,l}:=Var(\sum_{i=1}^{l}\xi_{n,i})$ for all $l=1,\dots,n,$ and $n\in\mathbb{N}$. Let furthermore $Q_{n,i}$ be the càdlàg inverse function of $t\mapsto P(|\xi_{n,i}|>t)$, i.e.

[TABLE]

with the convention that $\sup\emptyset:=0$. Let $\{\tilde{\alpha}_{n}(t):t\in\mathbb{N}\}$ be the sequence of mixing coefficients of $\{\xi_{n,i}:1\leq i\leq n,n\in\mathbb{N}\}$. For $t\in(0,\infty)$ define $\tilde{\alpha}_{n}(t):=\tilde{\alpha}_{n}(\left\lfloor t\right\rfloor)$. Let its càdlàg inverse function be defined by

[TABLE]

Condition (a) in Corollary 1 in Rio (1995) is easy to verify. Concerning condition (b) in aforementioned corollary note that by Markov’s inequality, it holds that for all $t>0$ and with $q:=Q\frac{2+\gamma}{2}$

[TABLE]

where $\tilde{M}:=(\sum_{j=1}^{K}|\lambda_{j}|)^{q}M$ for $M<\infty$ from assumption (U). Hence, $Q_{n,i}(u)\leq u^{-\frac{1}{q}}n^{-\frac{1}{2}}\tilde{M}^{\frac{1}{q}}$ holds for all $u>0$. By similar arguments, we obtain $\tilde{\alpha}_{n}^{-1}(u)\leq\tilde{A}-\log_{a}(u)$ for all $u>0$, where $\tilde{A}:=\log_{a}(A)$. Furthermore, $V_{n,n}$ converges to $\sum_{j_{1}=1}^{K}\sum_{j_{2}=1}^{K}\lambda_{j_{1}}\lambda_{j_{2}}(s_{j_{1}}\wedge s_{j_{2}})\Sigma(\bm{z}_{j_{1}}\wedge\bm{z}_{j_{2}})>0$. Putting the results together, it can be obtained that

[TABLE]

converges to zero, and thus condition (b) is satisfied.

Applying Corollary 1 in Rio (1995), it holds that $\sum_{i=1}^{n}\xi_{n,i}/(V_{n,n})^{1/2}$ converges to the standard normal distribution and thus the assertion follows. ∎

A.3 Proofs of lemmata

Proof of Lemma A.2.

For some $l$-times differentiable function $h:\bm{J}_{n}\to\mathbb{R}$ define the norm

[TABLE]

and the function class $\mathcal{H}:=\mathcal{C}_{1,n}^{l+\eta}(\bm{J}_{n}):=\{h:\bm{J}_{n}\to\mathbb{R}:\|h\|_{l+\eta}\leq 1,\sup_{\bm{x}\in\bm{J}_{n}}\left|h(\bm{x})\right|\leq z_{n}\sqrt{\log n}\}$ with $z_{n}:=q_{n}\delta_{n}((\log n)/(nh_{n}^{d}))^{1/2}$. The third bandwidth condition in (B2) implies

[TABLE]

and thus Lemma A.1 implies that $P(\hat{h}_{n}\in\mathcal{C}_{1,n}^{l+\eta}(\bm{J}_{n}))\to 1$ as $n\to\infty$ holds for $\hat{h}_{n}(\bm{x})=(m(\bm{x})-\hat{m}_{n}(\bm{x}))\omega_{n}(\bm{x})$. Let furthermore $\bm{X}_{t}\sim P$ and $\mathcal{F}:=\{\bm{x}\mapsto I\{\bm{x}\leq\bm{z}\}:\bm{z}\in\mathbb{R}^{d}\}$. Then the assertion of the lemma follows if we show

[TABLE]

To this end let $\varepsilon_{n1}=\varepsilon_{n2}=n^{-1/2}$ and $\varepsilon_{n3}=n^{-1/2}/(\log n)$ and let further $0=s_{1}<\dots<s_{K_{n}}=1$ partition $[0,1]$ in intervals of length $2\varepsilon_{n1}$ such that $K_{n}=O(\varepsilon_{n1}^{-1})$. Furthermore, let $J_{n}:=N_{[~{}]}\left(\varepsilon_{n2},\mathcal{F},\|\cdot\|_{L_{2}(P)}\right)$ and $M_{n}:=N_{[~{}]}\left(\varepsilon_{n3},\mathcal{H},\|\cdot\|_{\infty}\right)$, where $\|\cdot\|_{\infty}$ is the supremum norm on $\bm{J}_{n}$. Let $[\varphi_{1}^{l},\varphi_{1}^{u}],\dots,[\varphi_{J_{n}}^{l},\varphi_{J_{n}}^{u}]$ denote the brackets needed to cover $\mathcal{F}$. Note that they can be chosen to be indicator functions and therefore non negative. Let furthermore $[h_{1}^{l},h_{1}^{u}],\dots,[h_{M_{n}}^{l},h_{M_{n}}^{u}]$ define the brackets needed to cover $\mathcal{H}$. It can be shown that $J_{n}=O\left(\varepsilon_{n2}^{-2d}\right)$ and $M_{n}=O(\exp(c_{n}^{d}\varepsilon_{n3}^{-d/(l+\eta)}))$ and further

[TABLE]

In what follows we only consider the first line on the right hand side, while the other ones can be treated similarly. We apply Theorem 2.1 of Liebscher (1996) to the random variable (for $m,j,k$ fixed)

[TABLE]

The mixing coefficient of $\{Z_{t}:1\leq t\leq n\}$ can be bounded by the mixing coefficient of $\{\bm{X}_{t}:t\in\mathbb{Z}\}$ due to Bradley (1985), Section 2, remark (iv). Further, the variables are centered and have a bound of order $O(z_{n}\log n)$. Applying Theorem 2.1 to $\sum_{i=1}^{n}Z_{i}$ yields for all $\epsilon>0$ and $n\in\mathbb{N}$ large enough

[TABLE]

where the first and second bandwidth constraints in (B2) were used. Details are omitted for the sake of brevity. ∎

Proof of Lemma A.4.

First, using the uniform rates of convergence results in Lemma A.1 applied to $(m(\bm{X}_{t}),\bm{X}_{t})_{t\in\mathbb{Z}}$ together with the first and the last bandwidth condition in assumption (B2) as well as the second condition in assumption (B1), it can be shown that

[TABLE]

Defining the function class

[TABLE]

and imposing $\bm{X}_{t}\sim P$, the assertion of the lemma follows if we show

[TABLE]

[TABLE]

The proof of (A.2) uses similar techniques to those in the proof of Lemma A.2, while the proof of (A.2) follows applying Taylor expansion for $m$ and $f$. Details are given in the supplement. ∎

Proof of Lemma A.5.

First, using the uniform rates of convergence results in Lemma A.1 applied to $(U_{t},\bm{X}_{t})_{t\in\mathbb{Z}}$ together with assumptions on the bandwidth, it can be shown that

[TABLE]

Furthermore, it can be shown that uniformly in $\bm{z}\in\mathbb{R}^{d}$ and for $q:=Q\frac{2+\gamma}{2}>2$

[TABLE]

Defining the function class

[TABLE]

and imposing $(U_{t},\bm{X}_{t})\sim P$, the assertion of the lemma follows if we show

[TABLE]

The proof of (A.4) uses similar techniques as the proof of Lemma A.2. Details are given in the supplement. ∎

Proof of Lemma A.3.

It will be shown that uniformly in $s\in[0,1]$ and $\bm{z}\in\mathbb{R}^{d}$,

[TABLE]

To this end define the function class

[TABLE]

and for $s\in[0,1]$ and $\varphi\in\mathcal{F}$

[TABLE]

where $(U_{t},\bm{X}_{t})\sim P$ and $\int\varphi dP=0$. Similarly to the proof of Theorem 3.1 (ii) an application of Theorem 2.5 in Mohr (2018b) yields for all $\delta_{n}\searrow 0$ and with $d(\varphi,\psi):=\left\|\varphi-\psi\right\|_{L_{Q\frac{2+\gamma}{2}}(P)}$,

[TABLE]

Note that here the assumption $Q>(d+1)(2+\gamma)$ is needed as $\tilde{N}_{[~{}]}(\varepsilon,\mathcal{F},\|\cdot\|_{L_{2}(P)})=O(\varepsilon^{-2(d+1)})$. Then, for some fixed $\bm{z}\in\mathbb{R}^{d}$ defining $\varphi_{n}(u,\bm{x}):=uI\{\bm{x}\leq\bm{z}\}I\{\bm{x}\notin[-c_{n},c_{n}]^{d}\}$, it holds that $\varphi_{n}\in\mathcal{F}$ for all $n\in\mathbb{N}$ and

[TABLE]

where the convergence holds by the dominated convergence theorem as $\int c(\bm{x})^{Q}f(\bm{x})d\bm{x}<\infty$. With

[TABLE]

it can therefore be concluded that

[TABLE]

With (A.6) the last term is $o_{P}(1)$ which proves the assertion in (A.5) and therefore the assertion of the lemma. ∎

Supplementary material

Appendix B Additional simulation results

In addition to the models in section 5 in the main paper, in this section we investigate a second order autoregression model of the following form

[TABLE]

with $\Delta_{0}\in\{0,1.3\}$. We consider three different choices for the conditional variance function, namely $\sigma^{2}(x_{1},x_{2})=1$ for an AR(2) model, $\sigma^{2}(x_{1},x_{2})=1+0.4x_{1}^{2}$ for an AR(2)-ARCH(1) model and $\sigma^{2}(x_{1},x_{2})=1+0.2x_{1}^{2}+0.2x_{2}^{2}$ for an AR(2)-ARCH(2) model. In all cases $H_{0}$ is fulfilled for $\Delta_{0}=0$ and $H_{1}$ for $\Delta_{0}\neq 0$ with a change in the regression function occurring in $\left\lfloor n/2\right\rfloor$.

The limiting distribution from Corollary 3.2 applies, but as the covariate is multivariate we do not easily obtain an asymptotically distribution-free test. Thus we apply the bootstrap procedure from section 4. Table 2 shows the rejection frequencies for all three models, when using the tests based on $T_{n1}^{*}$ and $T_{n2}^{*}$, as well as the bootstrap versions of $KS$ and $CM$ under both $H_{0}$ and $H_{1}$. It can be seen that under $H_{0}$ the tests reject a little more often than in the models considered in section 5, overestimating the level of $5\%$ sometimes for finite sample sizes. Under the alternative the number of rejections increases rapidly for $T_{n1}^{*}$ and $T_{n2}^{*}$ with increasing $n$, while it stays relatively low for $KS$ and $CM$. In summary, the bootstrap tests perform reasonably well and are therefore an acceptable alternative to the tests using critical values of the limiting distribution, which are here not known due to multidimensional covariates. Furthermore in these models, they outperform the bootstrap versions of the CUSUM tests.

Appendix C Proofs of auxiliary results

In this section we give the proofs of (A.2) and (A.3) in Lemma A.4 as well as (A.4) in Lemma A.5.

Proof of (A.2) and (A.3) in Lemma A.4.

For the proof of (A.2) let $\varepsilon_{n}:=n^{-1/2}/(\log n)$ and $J_{n}:=N_{[~{}]}\left(\varepsilon_{n},\mathcal{F}_{n,1},\|\cdot\|_{L_{1}(P)}\right)$. It can be shown that there exists a partition $\bm{z}_{1},\dots,\bm{z}_{J_{n}}$ of $\mathbb{R}^{d}$ such that $\|\varphi_{j}^{u}-\varphi_{j}^{l}\|_{L_{1}(P)}\leq\varepsilon_{n}$ for all $j\in\{1,\dots,J_{n}\}$, where

[TABLE]

and

[TABLE]

It then holds that $J_{n}=O(\varepsilon_{n}^{-d})$. Using these brackets of $\mathcal{F}_{n,1}$, it can be shown that

[TABLE]

where it is sufficient to discuss the proof of

[TABLE]

as the other assertion works analogously. By defining

[TABLE]

and

[TABLE]

it holds that $\varphi_{j}^{u}(\bm{x})=\varphi_{j,1}^{u}(\bm{x})+\varphi_{j,2}^{u}(\bm{x})$. Thus, again the problem is reduced to showing

[TABLE]

the other assertion works analogously. Similar to before we apply Theorem 2.1 of Liebscher (1996) to the random variable $Z_{i}:=\varphi_{j,1}^{u}(\bm{X}_{i})-\int\varphi_{j,1}^{u}dP$ for fixed $j$. Note that the mixing properties are the same as the ones of the original process. Further the variables are centered and posses a bound of order $O(h_{n}q_{n})$. For all $\epsilon>0$ and $n\in\mathbb{N}$ large enough it can then be shown that

[TABLE]

where eventually the last bandwidth condition in assumption (B2) was used. The assertion in (A.3) can be shown by using Taylor’s expansion for both $m$ and $f$ up to order $r-1$ and the assumptions in (F1). Thus

[TABLE]

where the last equality holds by the third condition in (B2). ∎

Proof of (A.4) in Lemma A.5.

To this end let $\varepsilon_{n}:=n^{-1/2}/(\log n)$ and $J_{n}:=N_{[~{}]}\left(\varepsilon_{n},\mathcal{F}_{n,2},\|\cdot\|_{L_{1}(P)}\right)$. It can be shown that there exists a partition $\bm{z}_{1},\dots,\bm{z}_{J_{n}}$ of $\mathbb{R}^{d}$ such that $\|\varphi_{j}^{u}-\varphi_{j}^{l}\|_{L_{1}(P)}\leq\varepsilon_{n}$ for all $j\in\{1,\dots,J_{n}\}$, where

[TABLE]

where

[TABLE]

and similarly,

[TABLE]

where

[TABLE]

It then holds that $J_{n}=O(\varepsilon_{n}^{-}d)$. Using these brackets of $\mathcal{F}_{n,2}$, it can be shown that

[TABLE]

We will only consider the first term, as the second one is treated analogously. Similar to before we apply Theorem 2.1 of Liebscher (1996) to the random variable $Z_{i}:=\varphi_{j,1}^{u}(U_{i},\bm{X}_{i})-\int\varphi_{j,1}^{u}dP$ for fixed $j$. Note that the mixing properties are the same as the ones of the original process. Further, the variables are centered and posses a bound of order $O(n^{1/q})$. For all $\epsilon>0$ and $n\in\mathbb{N}$ large enough it can then be shown that

[TABLE]

due to the third bandwidth condition in assumption (B2) and because $q>2$. ∎