A Large Deviation Inequality for $\beta$-mixing Time Series and its Applications to the Functional Kernel Regression Model

TL;DR

This paper establishes a new large deviation inequality for $eta$-mixing time series, enabling improved concentration bounds and demonstrating its application in the consistency analysis of functional kernel regression models for dynamic forecasting.

Contribution

It introduces a novel large deviation inequality for $eta$-mixing processes and applies it to prove consistency in a functional kernel regression setting.

Findings

New large deviation inequality for $eta$-mixing time series

Application to prove consistency of functional kernel regression forecasts

Enhanced concentration inequalities for dependent data

Abstract

We give a new large deviation inequality for sums of random variables of the form $Z_k = f(X_k,X_t)$ for $k,t\in \mathbb{N}$, $t$ fixed, where the underlying process $X$ is $\beta$-mixing. The inequality can be used to derive concentration inequalities. We demonstrate its usefulness in the functional kernel regression model of Ferraty et al. (2007) where we study the consistency of dynamic forecasts.

Equations

$$\displaystyle\mathbbm{P}\left(n^{-1}\left|\sum_{k=1}^{n}f(X_{k},X_{t})-\mathbbm{E}\left[\,f(X_{0},x)\,\right]\big{|}_{x=X_{t}}\right|\geq\varepsilon\right)\text{ where }t\in\mathbbm{Z}$$

$$\alpha (\mathcal{F},\mathcal{G}):=\sup \left\{\left|\text{mathbbm}P(A\cap B)-\text{mathbbm}P(A)\text{mathbbm}P(B)\right|:A\in \mathcal{F},B\in \mathcal{G}\right\}.$$

$$\displaystyle\beta(\mathcal{F},\mathcal{G})\coloneqq\frac{1}{2}\sup\left\{\sum_{i\in I}\sum_{j\in J}\big{|}\mathbbm{P}(U_{i}\cap V_{j})-\mathbbm{P}(U_{i})\mathbbm{P}(V_{j})\big{|}\,:\,(U_{i})_{i\in I},(V_{j})_{j\in J}\text{ are finite partitions of }\Omega\right\}.$$

$$\displaystyle\alpha(n)\coloneqq\sup_{k\in\mathbbm{N}}\alpha\big{(}\sigma(X_{s}:s\leq k),\sigma(X_{s}:s\geq k+n)\big{)}\text{ and }\beta(n)\coloneqq\sup_{k\in\mathbbm{N}}\beta\big{(}\sigma(X_{s}:s\leq k),\sigma(X_{s}:s\geq k+n)\big{)}.$$

$$\beta (X,Y)={\Vert \text{mathbbm}{P}_{X\otimes Y}-\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y\Vert }_{TV}=\underset{A\in \mathcal{A}}{\sup }|\text{mathbbm}{P}_{X\otimes Y}(A)-\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y(A)|.$$

$$\text{mathbbm}{P}_{(X,Y)}\ll \text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y\text{ such that }g:=\frac{\mathrm{d}\text{mathbbm}{P}_{(X,Y)}}{\mathrm{d}(\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y)}\text{ satisfies }{\Vert g\Vert }_{\infty ,\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y}<\infty .$$

$$\left|{\int }_{S\times T}h\mathrm{d}\text{mathbbm}{P}_{(X,Y)}-{\int }_{S\times T}h\mathrm{d}(\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y)\right|\le 2^{1/q}(1+{\Vert g\Vert }_{\infty ,\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y}{)}^{1/p}{\Vert h\Vert }_{p,\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y}\beta (X,Y{)}^{1/q}.$$

$$\left|{\int }_{S\times T}h\mathrm{d}\text{mathbbm}{P}_{(X,Y)}-{\int }_{S\times T}h\mathrm{d}(\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y)\right|\le 2{\Vert h\Vert }_{\infty }\beta (X,Y).$$

$$\beta (X,Y)=\underset{A\in \mathcal{A}}{\sup }|\text{mathbbm}{P}_{X\otimes Y}(A)-\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y(A)|=\frac{1}{2}{\int }_{S\times T}|g-1|\mathrm{d}(\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y).$$

$$\left|{\int }_{S\times T}h\mathrm{d}\text{mathbbm}{P}_{(X,Y)}-{\int }_{S\times T}h\mathrm{d}(\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y)\right|\le {\int }_{S\times T}|h||g-1|\mathrm{d}(\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y)$$

$$\le {\left({\int }_{S\times T}|h{|}^p\mathrm{d}(\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y)\right)}^{1/p}{2}^{1/q}{\left(\frac{1}{2}{\int }_{S\times T}|g-1|\mathrm{d}(\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y)\right)}^{1/q}{\Vert g-1\Vert }_{\infty ,\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y}^{(q-1)/q}$$

$$\le 2^{1/q}(1+{\Vert g\Vert }_{\infty ,\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y}{)}^{1/p}{\Vert h\Vert }_{p,\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y}\beta (X,Y{)}^{1/q}.$$

$$\left|{\int }_{S\times T}h\mathrm{d}\text{mathbbm}{P}_{(X,Y)}-{\int }_{S\times T}h\mathrm{d}(\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y)\right|$$

$$\le {\Vert h\Vert }_{\infty }\left\{\sum _{i\in I}\text{mathbbm}{P}_{(X,Y)}(A_i)-\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y(A_i)+\sum _{i\in J}\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y(A_i)-\text{mathbbm}{P}_{(X,Y)}(A_i)\right\}$$

$$={\Vert h\Vert }_{\infty }\left\{\text{mathbbm}{P}_{(X,Y)}\left(\bigcup _{i\in I}{A}_i\right)-\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y\left(\bigcup _{i\in I}{A}_i\right)+\text{mathbbm}{P}_X\otimes \text{mathbbm}{P}_Y\left(\bigcup _{i\in J}{A}_i\right)-\text{mathbbm}{P}_{(X,Y)}\left(\bigcup _{i\in J}{A}_i\right)\right\}\le 2{\Vert h\Vert }_{\infty }\beta (X,Y).$$

$$\text{mathbbm}E\left[\exp (\gamma {\int }_{(0,A]}f(X_k,X_t))\right]\le 3{\kappa }_0\exp \left(-\frac{{\kappa }_{1}A}{4\log A}\right)+\exp \left(C{\gamma }^2{B}^{2}A\log A+\frac{\gamma BA}{\log A}\right).$$

$$\tilde{I}:=(-\delta A/2+t,\delta A/2+t]\cap (0,A].$$

$$\text{mathbbm}E\left[\exp \left(\gamma {\int }_0^{A}f(X_s,X_t)\mathrm{d}s\right)\right]$$

$$\le \text{mathbbm}E\left[\exp \left(\gamma {\int }_{I_L\cup I_R}f(X_s,X_t)\mathrm{d}s\right)\right]\exp \{\gamma B\delta A\}$$

$$\displaystyle\leq\Biggl{\{}\left|\mathbbm{E}\left[\,\exp\left(\gamma\int_{I_{L}\cup I_{R}}f(X_{s},X_{t})\,\mathrm{d}s\right)-\exp\left(\gamma\int_{I_{L}\cup I_{R}}f(X_{s},X^{\prime}_{0})\,\mathrm{d}s\right)\,\right]\right|$$

$$\displaystyle\quad+\mathbbm{E}\left[\,\exp\left(\gamma\int_{I_{L}\cup I_{R}}f(X_{s},X^{\prime}_{0})\,\mathrm{d}s\right)\,\right]\Biggl{\}}\,\exp\{\gamma B\delta A\}$$

$$\displaystyle\leq\Biggl{\{}2\beta\left(\frac{\delta A}{2}\right)\exp(\gamma B(1-\delta)A)+\alpha(\delta A)\exp(\gamma B(1-\delta)A)$$

$$\displaystyle\quad+\mathbbm{E}\left[\,\mathbbm{E}\left[\,\exp\left(\gamma\int_{I_{L}}f(X_{s},x)\,\mathrm{d}s\right)\,\right]\mathbbm{E}\left[\,\exp\left(\gamma\int_{I_{R}}f(X_{s},x)\,\mathrm{d}s\right)\,\right]\Bigg{|}_{x=X^{\prime}_{0}}\,\right]\Biggl{\}}\,\exp\{\gamma B\delta A\}$$

$$\displaystyle\begin{split}&\leq 3\kappa_{0}\exp\left(-\frac{\kappa_{1}\delta A}{2}+\gamma BA\right)\\ &\quad+\mathbbm{E}\left[\,\mathbbm{E}\left[\,\exp\left(\gamma\int_{I_{L}}f(X_{s},x)\,\mathrm{d}s\right)\,\right]\mathbbm{E}\left[\,\exp\left(\gamma\int_{I_{R}}f(X_{s},x)\,\mathrm{d}s\right)\,\right]\Bigg{|}_{x=X^{\prime}_{0}}\,\right]\,\exp\{\gamma B\delta A\}.\end{split}$$

$$\text{mathbbm}E\left[\exp \left(\gamma {\int }_{I_L}f(X_s,x)\mathrm{d}s\right)\right]\text{mathbbm}E\left[\exp \left(\gamma {\int }_{I_R}f(X_s,x)\mathrm{d}s\right)\right]$$

$$\le \exp \left(C{\gamma }^2{B}^{2}A\log A\right),$$

$$\text{mathbbm}E\left[\exp \left(\gamma {\int }_0^{A}f(X_s,X_t)\mathrm{d}s\right)\right]\le 3{\kappa }_0\exp \left(-\frac{{\kappa }_{1}A}{2\log A}+\gamma BA\right)+\exp \left(C{\gamma }^2{B}^{2}A\log A+\frac{\gamma BA}{\log A}\right).$$

$$\text{mathbbm}P\left(n^{-1}\left|\sum _{k=1}^n{f}_n(X_k,X_t)\right|\ge \epsilon \right)\le a_1\exp \left(-a_2\frac{\epsilon n}{B\log n\log \log n}\right).$$

$$\text{mathbbm}P\left(n^{-1}\left|\sum _{k=1}^n{f}_n(X_k,X_t)\right|\ge \epsilon \right)$$

$$\displaystyle\quad\cdot\Biggl{\{}3\kappa_{0}\exp\left(-\frac{\kappa_{1}n}{4\log n}\right)+\exp\left(C\gamma^{2}B^{2}n\log n+\frac{\gamma Bn}{\log n}\right)\Biggl{\}}.$$

$$\underset{k\in \text{mathbbm}N}{\sup }\Vert \frac{\mathrm{d}\text{mathbbm}{P}_{(X_0,X_k)}}{\mathrm{d}\left(\text{mathbbm}{P}_{X_0}\otimes \text{mathbbm}{P}_{X_k}\right)}\Vert <\infty .$$

$$\displaystyle\begin{split}&\mathbbm{P}\left(n^{-1}\left|\sum_{k=1}^{n}f_{n}(X_{k},X_{t})\right|\geq\varepsilon\right)\\ &\leq\inf_{B>1}\Bigg{\{}a_{1}\varepsilon^{-1}\exp\left(-a_{2}\frac{\varepsilon n}{B\log n\log\log n}\right)+4\varepsilon^{-1}(k-1)^{-1}B^{-(k-1)}\mathbbm{E}\left[\,f_{n}(X_{0},X^{\prime}_{0})^{k}\,\right]\\ &\quad+a_{1}(n\varepsilon)^{-1}\mathbbm{E}\left[\,f_{n}(X_{0},X^{\prime}_{0})^{pr}\,\right]^{1/(pr)}B^{-k/(pu)}\mathbbm{E}\left[\,f_{n}(X_{0},X^{\prime}_{0})^{k}\,\right]^{1/(pu)}\Bigg{\}}\end{split}$$

$$f_{n,+}(x,y):=f_n(x,y)-\min (f_n(x,y),B)\ge 0,f_{n,-}(x,y)$$

$$\text{ and }{f}_{n,0}(x,y):=\max (\min (f_n(x,y),B),-B).$$

$$\begin{array}{rl}\text{mathbbm}P\left(n^{-1}\left|\sum _{k=1}^n{f}_n(X_k,X_t)\right|>3\epsilon \right)& \le \text{mathbbm}P\left(n^{-1}\left|\sum _{k=1}^n{f}_{n,+}(X_k,X_t)-\text{mathbbm}E\left[f_{n,+}(X_k,x)\right]{|}_{x=X_t}\right|>\epsilon \right)\\ & +\text{mathbbm}P\left(n^{-1}\left|\sum _{k=1}^n{f}_{n,0}(X_k,X_t)-\text{mathbbm}E\left[f_{n,0}(X_k,x)\right]{|}_{x=X_t}\right|>\epsilon \right)\\ & +\text{mathbbm}P\left(n^{-1}\left|\sum _{k=1}^n{f}_{n,-}(X_k,X_t)-\text{mathbbm}E\left[f_{n,-}(X_k,x)\right]{|}_{x=X_t}\right|>\epsilon \right).\end{array}$$

$$\text{mathbbm}P\left(n^{-1}\left|\sum _{k=1}^n{f}_{n,+}(X_k,X_t)-\text{mathbbm}E\left[f_{n,+}(X_0,x)\right]{|}_{x=X_t}\right|>\epsilon \right)$$

$$\le (n\epsilon {)}^{-1}\sum _{k=1}^n\left|\text{mathbbm}E\left[\left|f_{n,+}(X_k,X_t)-\text{mathbbm}E\left[f_{n,+}(X_0,x)\right]{|}_{x=X_t}\right|-\left|f_{n,+}(X_k,X_0^{\prime })-\text{mathbbm}E\left[f_{n,+}(X_0,x)\right]{|}_{x=X_0^{\prime }}\right|\right]\right|$$

Full text

Keywords: Asymptotic inference; Asymptotic inequalities; $\beta$-mixing; Bernstein inequality; Nonparametric statistics; Time Series

MSC 2010: Primary: 62G20; 62M10; 37A25; Secondary: 62G09

1 Introduction

In this article we study asymptotic bounds for the probability

[TABLE]

for a real-valued function $f$ and for a stationary stochastic process $\{X_{k}:k\in\mathbbm{Z}\}$ which takes values in a state space $(S,\mathfrak{S})$. If the function $f$ is bounded by $B>1$, we obtain for a certain constant $c$ an exponential rate of convergence for (1.1) which is in $\mathcal{O}\left(\exp(-c\varepsilon n/(B\log n\log\log n)\,)\right)$. So modulo a logarithmic factor which comes from the dependence in the data, the rate corresponds to the rates of classical large deviations inequalities for independent random variables.

Large deviation inequalities are a major tool for the asymptotic analysis in probability theory and statistics. One of the first inequalities of this type was published by Bernstein (1927) who considers the case $\mathbbm{P}(|S_{n}|>\varepsilon)$, where $S_{n}=\sum_{k=1}^{n}X_{k}$ for bounded real-valued random variables $X_{1},\ldots,X_{n}$ which are i.i.d. and have expectation zero. There are various versions and generalizations of Bernstein’s inequality, e.g., Hoeffding (1963). In particular, deviation inequalities for dependent data such as stochastic processes are nowadays important: Bernstein inequalities for time series are developed in Carbon (1983), Collomb (1984), Bryc and Dembo (1996) and Merlevède et al. (2009). Arcones (1995) develops Bernstein-type inequalities for $U$-statistics. Valenzuela-Domínguez et al. (2017) give a further generalization to strong mixing random fields $\{X_{s}:s\in\mathbbm{Z}^{N}\}$ which are defined on the regular lattice $\mathbbm{Z}^{N}$ for some lattice dimension $N\in\mathbbm{N}_{+}$. A similar version for independent multivariate random variables is given by Ahmad and Amezziane (2013). Krebs (2017) gives a Bernstein inequality for strong mixing random fields which are defined on exponentially growing graphs. The corresponding definitions of dependence and their interaction properties can be found in Doukhan (1994) and in Bradley (2005). Bernstein inequalities often find their applications when deriving large deviation results or (uniform) consistency statements in nonparametric regression and density estimation, compare for instance Györfi et al. (2002).

Consider again Equation (1.1) where $X$ is a strong mixing process and $t\in\mathbbm{Z}$ is a fixed integer. In the special case that the aggregating function $f(x,y)=g(x)h(y)$ is multiplicative, the statement follows from the large deviation inequalities for strong mixing processes given e.g. in Merlevède et al. (2009). However, for a general aggregating function which does not allow a similar decomposition, we face the problem that the process given by $Z_{k}=f(X_{k},X_{t})$ is not necessarily strong mixing any more.

Thus, we need further assumptions on $X$ which ensure that the contribution of a single summand $f(X_{k},X_{t})$ is asymptotically negligible in order to derive the necessary bounds. We show that for $\beta$-mixing processes there exist such bounds under certain conditions which are quite likely to be fulfilled in practice.

In this manuscript we also give a natural motivation for the new large deviation inequality for (1.1) and we demonstrate its power in a non-trivial application to the functional kernel regression model. Here the regression operator, which maps from an infinite-dimensional space, e.g., a Hilbert space, to the real numbers, is estimated with a compactly supported kernel function. The literature on this topic is extensive and we refer the reader to Ferraty et al. (2007), Delsol (2009) and Ferraty et al. (2010) for a general introduction.

This paper is organized as follows: we give the definitions in Section 2. In Section 3 we present the new exponential inequalities for $\beta$-mixing time series. In Section 4 we apply the exponential inequalities in the nonparametric functional regression model of Ferraty et al. (2007) and investigate dynamic forecasts in this model.

2 Definitions and Notation

In this section we give the mathematical definitions and notation which we shall use to derive the results. We first introduce the $\alpha$- and $\beta$-mixing coefficients. The first coefficient is due to Rosenblatt (1956), the latter was introduced by Kolmogorov and Rozanov (1960):

Definition 2.1** ($\alpha$- and $\beta$-mixing coefficient).**

Let $(\Omega,\mathcal{A},\mathbbm{P})$ be a probability space. Given two sub-$\sigma$-algebras $\mathcal{F}$ and $\mathcal{G}$ of $\mathcal{A}$, the $\alpha$-mixing coefficient is defined by

[TABLE]

It is well-known that $\alpha(\mathcal{F},\mathcal{G})\leq 1/4$. The $\beta$-mixing coefficient is defined by

[TABLE]

Then $2\alpha(\mathcal{F},\mathcal{G})\leq\beta(\mathcal{F},\mathcal{G})\leq 1$, compare to Bradley (2005). If $X$ and $Y$ are two random variables on $(\Omega,\mathcal{A},\mathbbm{P})$, then we denote by $\alpha(X,Y)$ the mixing coefficient $\alpha(\sigma(X),\sigma(Y))$ and by $\beta(X,Y)$ the mixing coefficient $\beta(\sigma(X),\sigma(Y))$. Furthermore, if $(X_{k}:k\in\mathbbm{Z})$ is a stochastic process, then we write for $n\in\mathbbm{N}$

[TABLE]

The stochastic process $X$ is said to be strong mixing (or $\alpha$-mixing) if $\alpha(n)\rightarrow 0$ ($n\rightarrow\infty$). Furthermore, $X$ is $\beta$-mixing if $\beta(n)\rightarrow 0$ ($n\rightarrow\infty$). The $\beta$-mixing coefficient of two random variables $X,Y$ on $(\Omega,\mathcal{A},\mathbbm{P})$ is related to the total variation norm as (cf. Doukhan (1994))

[TABLE]

Moreover, we denote for a set $A\in\mathcal{A}$ by $\,\mathbbm{1}\!\left\{A\right\}$ the indicator function on $A$. If not stated otherwise, we agree on the convention to abbreviate constants by $C$ in the proofs. In the following we derive inequalities of the Bernstein-type for $\beta$-mixing stochastic processes.

3 Exponential inequalities for $\beta$-mixing processes

This section contains the main results. The following proposition is the first result of this article: we give inequalities for $\beta$-mixing processes which are similar to Davydov’s inequality (cf. Davydov (1968)).

Proposition 3.1** ($\beta$-mixing and integration w.r.t. the joint distribution).**

Let $(\Omega,\mathcal{A},\mathbbm{P})$ be a probability space and let $(S,\mathfrak{S})$ and $(T,\mathfrak{T})$ be measurable, topological spaces. Let $(X,Y)\colon\Omega\rightarrow S\times T$ be $\mathfrak{S}\otimes\mathfrak{T}$-measurable such that the joint distribution of $X$ and $Y$ is absolutely continuous w.r.t. their product measure on $\mathfrak{S}\otimes\mathfrak{T}$ with an essentially bounded Radon-Nikodým derivative $g$, i.e.,

[TABLE]

Let $p,q\in[1,\infty]$ be Hölder conjugate, i.e., $p^{-1}+q^{-1}=1$. Let $h\colon S\times T\rightarrow\mathbbm{R}$ be measurable and $p$-integrable w.r.t. the product measure, i.e., $\left\lVert h\right\rVert_{p,\mathbbm{P}_{X}\otimes\mathbbm{P}_{Y}}=\mathbbm{E}\left[\,\mathbbm{E}\left[\,|h(X,y)|^{p}\,\right]|_{y=Y}\,\right]^{1/p}<\infty$. Then

[TABLE]

If $p=\infty$ and $q=1$ and if $\left\lVert h\right\rVert_{\infty}=\sup_{(x,y)\in S\times T}|h(x,y)|<\infty$, this statement is true without the assumption on the absolute continuity of the distributions:

[TABLE]

Proof of Proposition 3.1.

We begin with the first statement. Therefore we shall use (2.1) and the following fact: since the joint distribution is absolutely continuous, the total variation (and the $\beta$-mixing coefficient) can be written as

[TABLE]

Indeed, this well-known equality follows if one considers the sets $\{g<1\},\{g>1\}\in\mathcal{A}$. Thus, Equation (3.3) and the Hölder inequality enable us to write

[TABLE]

We come to the second statement. Let $p=\infty$ and $q=1$. Let $h$ be bounded as $\sup_{(x,y)\in S\times T}|h(x,y)|<\infty$ and assume w.l.o.g. that $h$ is non negative. Such a function $h$ can be approximated uniformly on $S\times T$ by a sequence of simple functions $h_{n}$ which converge from below to $h$, i.e., $0\leq\sup_{(x,y)\in S\times T}h(x,y)-h_{n}(x,y)\leq 2^{-n}$. Hence, it suffices to consider simple functions: let $h=\sum_{i=1}^{N}a_{i}\,\mathbbm{1}\!\left\{A_{i}\right\}$ for $N\in\mathbbm{N}$ and real numbers $a_{i}\in\mathbbm{R}_{+}$ and pairwise disjoint sets $A_{i}\in\mathfrak{S}\otimes\mathfrak{T}$. Set $I\coloneqq\{i:1\leq i\leq N,\mathbbm{P}_{X}\otimes\mathbbm{P}_{Y}(A_{i})\leq\mathbbm{P}_{(X,Y)}(A_{i})\}$ and $J\coloneqq\{1,\ldots,N\}\setminus I$. Hence, $I$ (respectively $J$) is the index sets where the probability of the joint distribution exceeds (respectively is less than) the probability of the product measure. Then, the second statement follows:

[TABLE]

∎

We present the main results of this article: we derive the exponential inequalities which prove the convergence of the sum n^{-1}\sum_{k=1}^{n}f(X_{k},X_{t})-\mathbbm{E}\left[\,f(X_{0},x)\,\right]\big{|}_{x=X_{t}} for some $t\in\mathbbm{Z}$ fixed. We assume for the rest of the article that the $\beta$-mixing coefficients (and the $\alpha$-mixing coefficients) of $X$ are exponentially decreasing, i.e., there are $\kappa_{0},\kappa_{1}\in\mathbbm{R}_{+}$ such that $\beta(n)\leq\kappa_{0}\exp(-\kappa_{1}n)$. Dedecker and Prieur (2004) give an example of such processes: consider a stationary Markov chain $X_{n}=\psi(X_{n-1})+\varepsilon_{n}$ where the innovations are integrable and i.i.d. and $\psi$ is Lipschitz-continuous with a Lipschitz constant strictly smaller than 1. Then the chain is geometrically $\beta$-mixing if the distribution of the innovations has an absolutely continuous component which is strictly positive in a neighborhood of zero. For more details, we refer to Dedecker and Prieur (2004) and the references therein.

Moreover, we assume w.l.o.g. for the next results that the probability space contains an additional random variable $X^{\prime}_{0}$ which has the same marginals as the stationary process $X$ but is independent of $X$. We follow the ideas of Merlevède et al. (2009) who investigate sums of $\alpha$-mixing processes. Therefore, we define an extension of the discrete process $X=\{X_{k}:k\in\mathbbm{Z}\}$ to the real interval: define for a real number $t$ the random variable $X_{t}$ by $X_{\lfloor t\rfloor}$ which makes the process $\{X_{t}:t\in\mathbbm{R}\}$ right continuous. In the same way, we extend the mixing coefficients to the real numbers, i.e., $\alpha(t)=\alpha(\lfloor t\rfloor)$ and $\beta(t)=\beta(\lfloor t\rfloor)$ for $t\in\mathbbm{R}$. It follows from this definition of the continuous time process that $\sum_{k=1}^{n}f(X_{k},X_{t})=\int_{1}^{n+1}f(X_{s},X_{t})\,\mathrm{d}s$. We present the main result of this section.

Theorem 3.2**.**

Let $\{X_{k}:k\in\mathbbm{Z}\}$ be a stationary $\beta$-mixing process whose marginals $X_{k}$ take values in a state space $(S,\mathfrak{S})$. The $\beta$-mixing coefficients satisfy $\beta(n)\leq\kappa_{0}\exp(-\kappa_{1}n)$ for some $\kappa_{0},\kappa_{1}\in\mathbbm{R}_{+}$. Let $f\colon S\times S\rightarrow[-B,B]$, $B\in\mathbbm{R}_{+}$, be a bounded and measurable function which fulfills $\mathbbm{E}\left[\,f(X_{s},x)\,\right]=0$ for $\mathbbm{P}_{X_{0}}$-almost all $x\in S$. Let $A\geq 14\vee(2\kappa_{1})$. Then there is a constant $C$ which only depends on $\kappa_{0}$ and $\kappa_{1}$ such that for all $0<\gamma B\leq\frac{1\wedge\kappa_{1}}{2}\wedge\frac{\kappa_{1}}{4\log A}$

[TABLE]

Proof of Theorem 3.2.

Let $t\in\mathbbm{R}$ be fixed. In the first step we divide the interval $(0,A]$ into three pairwise disjoint intervals $I_{L},\tilde{I}$ and $I_{R}$. Therefore, let $\delta\coloneqq 1/\log A\in(0,1)$, then $\tilde{I}$ is given by

[TABLE]

$I_{L}$ is then given as the left residual part of $I$, namely, $I_{L}=(-\infty,-\delta/2A+t]\cap(0,A]$. In the same way, $I_{R}$ is the right residual part, $I_{R}=(\delta/2A+t,\infty]\cap(0,A]$. Note that both $I_{L}$ and $I_{R}$ have a minimal length of $(1-\delta)A/2$ and that their measure sums up to at most $A$. Furthermore, for all $A\geq 14$, $I_{R}$ and $I_{L}$ have a length of at least 4. In order to estimate the Laplace transform, we bound the integral over the interval $\tilde{I}$ by its maximal value and then apply Proposition 3.1 and Proposition A.1:

[TABLE]

Consider the inner expectations in (LABEL:EqLaplace1): since the random variable $f(X_{s},x)$ is centered for $\mathbbm{P}_{X_{0}}$-almost all $x\in S$, we can apply Lemma 10 from Merlevède et al. (2009) and obtain if $(1-\delta)A/2>4\vee(2\kappa_{1})$ that

[TABLE]

where $0\leq\gamma B\leq(\kappa_{1}\wedge 1)/2$ and $C$ is a constant which only depends on the bound of the mixing coefficients $\kappa_{0},\kappa_{1}$. Combining (LABEL:EqLaplace1) with (3.5) yields

[TABLE]

∎

Since the bound $B$ is the only property of the function $f$ which is determining the bound on the Laplace transform given in Theorem 3.2, we can easily extend the statement to a sequence of functions $(f_{n}:n:\mathbbm{N})$ which are all uniformly bounded: $f_{n}\colon S\times S\rightarrow[-B,B]$ for $n\in\mathbbm{N}$. Therefore, we give the corollary

Corollary 3.3**.**

Let the stochastic process $X$ be as in Theorem 3.2. Let $(f_{n}:n:\mathbbm{N})$ be a sequence of functions such that each element $f_{n}\colon S\times S\rightarrow[-B,B]$ is as in Theorem 3.2 and satisfies in particular $\mathbbm{E}\left[\,f_{n}(X_{0},x)\,\right]=0$ for $\mathbbm{P}_{X_{0}}$-almost all $x\in S$. Then for all $n\in\mathbbm{N}$ there are constants $a_{1},a_{2}$ which only depend on the parameter $\kappa_{0},\kappa_{1}$ such that

[TABLE]

Proof of Corollary 3.3.

If we apply Theorem 3.2 to the interval $(0,n]$ and to the process $X$ which is formally shifted one integer to the left, we find with the choice $\gamma=(\kappa_{1}\wedge 1)/(4B\log n\log\log n)$ and Markov’s inequality that for $n\geq 14$

[TABLE]

Note that the last term in the curly brackets grows at a rate of $n/\left(\log n(\log\log n)^{2}\right)$. This finishes the proof. ∎

We consider the special case where $f_{n}(X,Y)\equiv X$ in the context of Corollary 3.3. We obtain in this case for a $\beta$-mixing process $X$ that $\mathbbm{P}\left(\left|n^{-1}\sum_{k=1}^{n}X_{k}\right|\geq\varepsilon\right)\leq a_{1}\exp\left\{-a_{2}\varepsilon\,n/(\log n\log\log n)\right\}$. This corresponds to the rate given by Merlevède et al. (2009) who investigate concentration inequalities of centered, real-valued and strongly mixing processes $X$ where the $\alpha$-mixing coefficients decay at an exponential rate. This means that our new inequality from Corollary 3.3 attains the same rate in this special case.

We can derive an extension of Theorem 3.2 to a sequence of unbounded functions $f_{n}\colon S\times S\rightarrow\mathbbm{R}$.

Theorem 3.4**.**

Let the stationary process $X$ be as in Theorem 3.2. Furthermore, let $(f_{n}:n\in\mathbbm{N})$ be a sequence of functions $f_{n}\colon S\times S\rightarrow\mathbbm{R}$ which fulfills $\mathbbm{E}\left[\,f_{n}(X_{0},x)\,\right]=0$ for $\mathbbm{P}_{X_{0}}$-almost all $x\in S$. Let the joint distribution of $(X_{0},X_{k})$ be absolutely continuous w.r.t. the product measure such that the corresponding Radon-Nikodým derivatives are essentially bounded uniformly over $k\in\mathbbm{N}$, i.e.,

[TABLE]

Then, there are constants $a_{1},a_{2}\in\mathbbm{R}_{+}$ which do not depend on $n\in\mathbbm{N}$ and $t\in\mathbbm{Z}$ such that for all $\varepsilon>0$

[TABLE]

where $p>1$, $r^{-1}+u^{-1}=1,r,u>1$ and $k>1$.

Proof of Theorem 3.4.

If the right hand side of (LABEL:EqUnboundedExponentialInequality0) is infinite, there is nothing to prove. So we can assume that the parameters $k,p,r$ and $u$ are such that the all moments on the right hand side exist. Since $\mathbbm{E}\left[\,f_{n}(X_{0},x)\,\right]=0$ for $\mathbbm{P}_{X_{0}}$-almost all $x\in S$, we use for $B>1$ the fundamental decomposition $f_{n}(x,y)=f_{n,+}(x,y)+f_{n,0}(x,y)+f_{n,-}(x,y)$ where

[TABLE]

Then,

[TABLE]

The asymptotic behavior of the second probability in (3.7) is given in Corollary 3.3. The first and the third probability can be bounded with Markov’s inequality and Proposition 3.1, we only consider $f_{n,+}$ here:

[TABLE]

where $p,q>1$ such that $p^{-1}+q^{-1}=1$ as in Equation (3.1) and where we use in the last inequality that the $\beta$-mixing coefficients are summable. We bound the two expectations given in (LABEL:UnboundedExponentialInequality2) further. For the first expectation we obtain for $k>1$

[TABLE]

The second expectation can be bounded with Hölder’s inequality for $r^{-1}+u^{-1}=1,r,u>1$ and Markov’s inequality:

[TABLE]

for $k>1$. This proves the claim. ∎

4 An application in the nonparametric functional regression model

In this section, we embed the developed inequalities in the nonparametric functional kernel regression model of Ferraty et al. (2007), Delsol (2009) and Ferraty et al. (2010). We cannot discuss all details of this model here due to its complexity and assume that the reader has some prior knowledge on this subject. Nevertheless, we describe all necessary assumptions: let $(\mathcal{H},\left\langle\cdot,\cdot\right\rangle)$ be a separable Hilbert space over $\mathbbm{R}$ where the norm $\left\lVert\,\cdot\,\right\rVert$ is induced by the inner product $\left\langle\cdot,\cdot\right\rangle$. In typical applications the Hilbert space is given as a function space $L^{2}([0,1],\mathcal{B}([0,1]),\,\mathrm{d}x)$ or more general $L^{2}(K,\mathcal{B}(K),\nu)$ for a bounded and convex set $K\in\mathbbm{R}^{d}$ and a finite measure $\nu$. Let $\{(X_{k},Y_{k}):k\in\mathbbm{Z}\}\subseteq\mathcal{H}\times\mathbbm{R}$ be a stationary process which is $\beta$-mixing with exponentially decaying coefficients. The process fulfills the regression equation

[TABLE]

The regression function $\psi\colon\mathcal{H}\rightarrow\mathbbm{R}$ is Lipschitz-continuous, i.e., $|\psi(x)-\psi(y)|\leq L\left\lVert x-y\right\rVert$ for all $x,y\in\mathcal{H}$. The error terms fulfill $\mathbbm{E}\left[\,\varepsilon_{k}|X_{k}\,\right]=0$ and $\mathbbm{E}\left[\,\varepsilon_{k}^{2}|X_{k}\,\right]<\infty$. Given an observed sample $(X_{1},Y_{1}),\ldots,(X_{n},Y_{n})$ the estimator of $\psi$ is

[TABLE]

where $h>0$ is the bandwidth and $K$ is a kernel function. The kernel function is supported in $[0,1]$ and zero otherwise. It admits a continuous derivative $K^{\prime}$ on $[0,1)$ such that $K^{\prime}\leq 0$ and $K(1)>0$ as in Ferraty et al. (2007). Moreover, we define the quantities

[TABLE]

where $F_{x}(h)=\mathbbm{P}(\left\lVert X_{0}-x\right\rVert\leq h)$. Note that we do not normalize $\hat{f}_{h}$ and $\hat{g}_{h}$ by a division with the bandwidth instead we multiply the inverse of the small ball probability $F_{x}(h)$. Here we assume that $F_{x}(h)>0$ for $h>0$ and that $F_{x}(0)=0$ for all $x\in\mathcal{H}$. We choose the bandwidth $h$ as a function of $n$ such that for $h\rightarrow 0$ as $n\rightarrow\infty$ the summability condition

[TABLE]

is fulfilled where we assume that $\mathbbm{E}\left[\,F_{X_{0}}(h)^{-2}\,\right]$ is finite for each $h>0$. E.g. we can choose $h\rightarrow 0$ such that $n^{-2}\mathbbm{E}\left[\,F_{X_{0}}(h)^{-2}\,\right]=o\left(n^{-(1+\delta)}\right)$ for some $\delta\in(0,1)$. Note that for the pointwise convergence $\hat{\psi}(x)\rightarrow\psi(x)$ Ferraty et al. (2007) require $n^{-1}F_{x}(h)^{-1}\rightarrow 0$.

Assume that there is a small ball probability function $\tau:[0,1]\rightarrow[0,1]$ which the uniform limit of the fractions $F_{x}(hs)/F_{x}(h)$, i.e.,

[TABLE]

Further, set $M=K(1)-\int_{0}^{1}K^{\prime}(s)\tau(s)\,\mathrm{d}s$ which is assumed to be positive.

We shortly discuss the issue of the convergence of $F_{x}(hs)/F_{x}(h)$ for $h\downarrow 0$ and $s\in(0,1)$, a more detailed introduction to this topic offer Ferraty et al. (2007) and the references therein. If the underlying Hilbert space is a function space, then one has in many applications that for a point $x$ in the Hilbert space $\mathbbm{P}(\left\lVert X_{0}-x\right\rVert<h)\sim C(x)\mathbbm{P}(\left\lVert X_{0}\right\rVert<h)$ for $h\downarrow 0$. Hence the small ball problem at the point $x$ shifts to the origin and in the limit the quotient $F_{x}(hs)/F_{x}(h)$ becomes independent of the point $x$. This motivates assumption (4.4).

We study the dynamic forecast $\hat{\psi}(X_{t})$ for an observation $X_{t}$, this means, we give sufficient conditions that the difference $\hat{\psi}(X_{t})-\psi(X_{t})$ converges to zero $a.s.$ We can use Theorem 3.2 and Theorem 3.4 for this problem and do not need to consider the difference $\hat{\psi}-\psi$ pointwise for $x\in\mathcal{H}$. We have the following theorem

Theorem 4.1**.**

Let $(X,\varepsilon)=\{(X_{k},\varepsilon_{k}):k\in\mathbbm{Z}\}\subseteq\mathcal{H}\times\mathbbm{R}$ be a stochastic process as in Theorem 3.4. Let $\{\varepsilon_{k}:k\in\mathbbm{Z}\}$ be a sequence of innovations such that the regression model (4.1) is fulfilled. Let (4.3) be satisfied for $h\rightarrow 0$ as $n\rightarrow\infty$. Then for a component $X_{t}$ of the process ($t\in\mathbbm{Z}$) it is true that $\hat{\psi}(X_{t})\rightarrow\psi(X_{t})$ a.s as $n\rightarrow\infty$.

Proof of Theorem 4.1.

We show that under the conditions we have both $\hat{f}_{h}(X_{t})\rightarrow M$ $a.s.$ and $\hat{g}_{h}(X_{t})\rightarrow\psi(X_{t})M$ $a.s.$ We begin with $\hat{f}_{h}(X_{t})$. We have that \left|\hat{f}_{h}(X_{t})-\mathbbm{E}\left[\,\hat{f}_{h}(x)\,\right]\Big{|}_{x=X_{t}}\right|\rightarrow 0 $a.s.$ Indeed, let $\varepsilon>0$ be arbitrary but fixed, we apply Equation (LABEL:EqUnboundedExponentialInequality0) from Theorem 3.4 with the following parameters

[TABLE]

Note that $f_{n}(X_{0},X^{\prime}_{0})$ corresponds to the function $K\left(\left\lVert X_{0}-X^{\prime}_{0}\right\rVert/h\right)/F_{X^{\prime}_{0}}(h)$ in this case. An application of the theorem of Fubini-Tonelli yields

[TABLE]

by the uniform convergence of the small probability from (4.4). Hence, we obtain for some constants $a_{1},a_{2}\in\mathbbm{R}_{+}$ that

[TABLE]

Consequently, the probabilities from the left hand side of (4.5) are summable over $n\in\mathbbm{N}_{+}$ for $\varepsilon>0$ fixed because of the choice of the bandwidth from (4.3). Thus, the claim that \hat{f}_{h}(X_{t})-\mathbbm{E}\left[\,\hat{f}_{h}(x)\,\right]\Big{|}_{x=X_{t}}\rightarrow 0 $a.s.$ follows with an application of the first Borel-Cantelli Lemma. An application of the theorem of Fubini-Tonelli yields

[TABLE]

The last inequality follows similarly as in Ferraty et al. (2007) and from the requirement (4.4). This proves that $\hat{f}_{h}(X_{t})\rightarrow M>0$ $a.s.$ Consider next $\hat{g}_{h}(X_{t})$. Once more, we have \left|\hat{g}_{h}(X_{t})-\mathbbm{E}\left[\,\hat{g}_{h}(x)\,\right]\Big{|}_{x=X_{t}}\right|\rightarrow 0 $a.s.$ using the requirement (4.3). Furthermore, we obtain for a point $x\in\mathcal{H}$ with the assumption that the regression function $\psi$ is Lipschitz continuous and that the conditional expectation of the innovations is zero

[TABLE]

This ensures that \left|\mathbbm{E}\left[\,\hat{g}_{h}(x)\,\right]\big{|}_{x=X_{t}}-M\psi(X_{t})\right|\rightarrow 0 $a.s.$ and proves the second statement $\hat{g}_{h}(X_{t})\rightarrow\psi(X_{t})M$ $a.s.$

All in all, we have $\hat{\psi}(X_{t})-\psi(X_{t})=\hat{g}_{h}(X_{t})/\hat{f}_{h}(X_{t})-\psi(X_{t})\rightarrow 0$ $a.s.$ by the continuous mapping theorem. ∎

Appendix A Appendix

The following statement can be seen as a multivariate generalization of Davydov (1968) in a special case:

Proposition A.1** (Ibragimov (1962)).**

Let $Z_{1},\ldots,Z_{n}$ be real-valued non-negative random variables each $a.s.$ bounded. Denote by $\alpha\coloneqq\sup_{k\in\{1,\ldots,n\}}\alpha\left(\sigma(Z_{i}:i\leq k),\sigma(Z_{i}:i>k)\right)$. Then

[TABLE]

Acknowledgments

The author is very grateful to a referee for thoughtful suggestions and comments which significantly improved and clarified the manuscript.