Asymptotics of an empirical bridge of regression on induced order statistics

TL;DR

This paper introduces a new statistical test for linear regression models based on empirical bridges derived from induced order statistics, with proven convergence properties and chi-square type tests.

Contribution

It presents a novel class of tests using empirical bridges for regression on concomitants, with theoretical proof of weak convergence to Gaussian processes.

Findings

Empirical bridges converge weakly to Gaussian processes.

Tests are of chi-square type and based on residual sums.

Method provides a new approach for regression analysis on induced order statistics.

Abstract

We propose a class of tests for linear regression on concomitants (induced order statistics). These tests are based on sequential sums of regression residuals. We self-center and self-normalize these sums. The resulting process is called an empirical bridge. We prove weak convergence of the empirical bridge in uniform metrics to a centered Gaussian process. The proposed tests are of chi-square type.

Equations

$${\xi }_i\theta \stackrel{def}{=}\sum _{j=1}^m{\xi }_{ij}{\theta }_j,$$

$$\hat{\theta }=(X^{T}X{)}^{-1}{X}^{T}Y.$$

$$b^2(x)=\mathbf{E}\left(({\xi }_1-h(x){)}^T({\xi }_1-h(x))|{\delta }_1=x\right)$$

$$\left(\frac{k}{n},\frac{{\hat{\Delta }}_k}{\sigma \sqrt{n}}\right).$$

$$K(s,t)=\min (s,t)-L(s)G^{-1}{L}^T(t),s,t\in [0,1].$$

$$Z_n^0(t)=\frac{\sigma }{\hat{\sigma }}(Z_n(t)-t{Z}_n(1)),0\le t\le 1,$$

$$L_{n,j}(t)=\frac{1}{n}\sum _{i=1}^{[nt]}{X}_{ij},$$

$$K^0(s,t)=\min \{s,t\}-st-L^0(s)G^{-1}(L^0(t){)}^T,s,t\in [0,1].$$

$$\mathbf{q}=(Z_n^0(1/(d+1)),\dots ,Z_n^0(d/(d+1))),$$

$${\hat{K}}^0(s,t)=\min (s,t)-st-L_n^0(s){)}^T{\hat{G}}^{-1}(L_n^0(t){)}^T,$$

$$\min (s,t)-L_1(s)L_1(t)/\mathbf{E}{\xi }_{11}^2.$$

$$\min (s,t)-st-L_1^0(s)L_1^0(t)/\mathbf{Var}{\xi }_{11}.$$

$${\hat{\Delta }}_k=\sum _{i=1}^k(Y_i-X_i\hat{\theta })=\sum _{i=1}^k(X_i(\theta -\hat{\theta })+{\epsilon }_i)$$

$$=\sum _{i=1}^k(X_i(\theta -(X^{T}X{)}^{-1}{X}^{T}Y)+{\epsilon }_i)$$

$$=\sum _{i=1}^k(X_i(\theta -(X^{T}X{)}^{-1}{X}^T(X\theta +\epsilon ))+{\epsilon }_i)$$

$$=\sum _{i=1}^k({\epsilon }_i-X_i(X^{T}X{)}^{-1}{X}^T\epsilon ).$$

$$\left\{\sum _{i=1}^{[nt]}({\epsilon }_i-L(t)G^{-1}{X}^T\epsilon ),t\in [0,1]\right\}.$$

$$\left\{\sum _{i=1}^{[nt]}(X_i{\epsilon }_i,{\epsilon }_i),t\in [0,1]\right\}.$$

$$\mathbf{E}({\xi }_1{e}_1|{\delta }_1=x)=0,\mathbf{E}(e_1|{\delta }_1=x)=0,x\in [0,1].$$

$$\tilde{b}^{2}(x)={\bf E}\left((\xi_{1}e_{1},e_{1})^{T}(\xi_{1}e_{1},e_{1})\ |\ \delta_{1}=x\right)=\sigma^{2}\left(\begin{array}[]{cc}b^{2}(x)+h^{T}(x)h(x)&h^{T}(x)\\ h(x)&1\end{array}\right).$$

$$\tilde{b}(x)=\sigma\left(\begin{array}[]{cc}b(x)&h^{T}(x)\\ {\bf 0}&1\end{array}\right).$$

$$\left\{\frac{1}{\sqrt{n}}{\left(\sum _{i=1}^{[nt]}{X}_i{\epsilon }_i,\sum _{i=1}^{[nt]}{\epsilon }_i\right)}^T,t\in [0,1]\right\}$$

$$\left\{{\int }_0^t\tilde{b}(x)d{\mathbf{W}}_{m+1}(x),t\in [0,1]\right\}.$$

$$\left\{\frac{1}{\sigma \sqrt{n}}\sum _{i=1}^{[nt]}({\epsilon }_i-L(t)G^{-1}{X}^T\epsilon {)}^T,t\in [0,1]\right\}$$

$$Z(t)=W_{m+1}(t)-L(t)G^{-1}{\left({\int }_0^{1}b(x)d{\mathbf{W}}_m(x)+{\int }_0^1{h}^T(x)d{W}_{m+1}(x)\right)}^T,$$

$$K(s,t)=\mathbf{E}Z(s)Z(t)$$

$$=\min (s,t)-L(s)G^{-1}{\int }_0^t{h}^T(x)dx-L(t)G^{-1}{\int }_0^s{h}^T(x)dx$$

$$+L(s)G^{-1}{\int }_0^1(b^2(x)+h^T(x)h(x))dx{G}^{-1}{L}^T(t)$$

$$=\min (s,t)-L(s)G^{-1}{L}^T(t).$$

Full text

Keywords: concomitants, weak convergence, regression residuals, empirical bridge.

1 Introduction

There are few approaches to regression models testing. An empirical fluctuation process function is implemented in R package by Zeileis at al. [40]. The process uses recursive regression residuals proposed by Brown, Durbin, Ewans [11].

MacNeill [31] studied linear regression for time series. He obtained limit processes for sequences of partial sums of regression residuals. Later Bischoff [10] showed that the MacNeill’s theorem holds in a more general setting. Aue et al. [1] introduced a new test for polynomial regression functions which is analogous to the classical likelihood test. This approach is developed in [2], [3].

Stute [37] proposed a class of tests for one-parametric case.

Our approach is adopted specifically for regression on induced order statistics (concomitants). These models arise in applications [28]. Partial results are proposed in [29], [30]. The proofs use the theory of induced order statistics.

David [12] and Bhattacharya [7] have introduced induced order statistics (concomitants) simultaneously. The asymptotic theory was developed in in [4], [5], [6], [8], [9], [13], [14], [15], [16], [17], [18], [19], [21], [22], [24], [38], [39]. Strong convergence to a corresponding Gaussian process can be proved by methods of [20], [27], [32], [35].

New contributions to the theory deal with extremal order statistics [23], [34], [36].

2 Main result and corollaries

Let $(\delta_{i},\xi_{i},\eta_{i})=(\delta_{i},\xi_{i1},\ldots,\xi_{im},\eta_{i})$ be independent and identically distributed random vector rows, $\delta_{i}$ has uniform distribution on $[0,1]$, $i=1,\ldots,n$. Random variables $\delta_{1},\ldots,\delta_{n}$ are not observed. They will be used for ordering.

We assume a linear regression hypothesis $\eta_{i}=\xi_{i}\theta+e_{i}$. Here

[TABLE]

$\{e_{i}\}_{i=1}^{n}$ and $\{(\delta_{i},\xi_{i})\}_{i=1}^{n}$ are independent, $\{e_{i}\}_{i=1}^{n}$ are i.i.d., ${\bf E}\,e_{1}=0$, ${\bf Var}\,e_{1}=\sigma^{2}>0$.

Vector $\theta=(\theta_{1},\ldots,\theta_{m})$ and constant $\sigma^{2}$ are unknown.

We order rows by the first component, that is, we change rows $j$ and $k$ while $j<k$ and $\delta_{j}>\delta_{k}$. The result is matrix $(U,X,Y)$ with rows $(U_{i},{\bf X}_{i},Y_{i})=(U_{i},X_{i1},\ldots,X_{im},Y_{i})$. So $U_{1}<\ldots<U_{n}$ a.s. are order statistics from uniform distribution on $[0,1]$. Elements of matrix $(X,Y)$ are concomitants. Let $\varepsilon_{i}=Y_{i}-{\bf X}_{i}\theta$. Note that $\varepsilon_{i}$ are i.i.d. and have the same distribution as $e_{1}$. Sequences $\{\varepsilon_{i}\}_{i=1}^{n}$ and $\{(U_{i},X_{i})\}_{i=1}^{n}$ are independent.

Let $\widehat{\theta}$ be LSE:

[TABLE]

It does not depend on the order of rows.

Let $h(x)={\bf E}\{\xi_{1}|\delta_{1}=x\}$ be conditional expectation, $L(x)=\int\limits_{0}^{x}h(s)\,ds$ be induced theoretical generalised Lorentz curve (see [1]),

[TABLE]

be a matrix of conditional covariances.

Let $g_{ij}={\bf E}\xi_{1i}\xi_{1j}$,  $G=(g_{ij})_{i,j=1}^{m}$, Then $G=\int_{0}^{1}(b^{2}(x)+h^{T}(x)h(x))\,dx$.

Let $\widehat{\varepsilon}_{i}=Y_{i}-X_{i}\widehat{\theta}$, $\widehat{\Delta}_{k}=\sum\limits_{i=1}^{k}\widehat{\varepsilon}_{i}$, $\widehat{\Delta}_{0}=0$.

Let $Z_{n}=\{Z_{n}(t),\,0\leq t\leq 1\}$ be a piecewise linear random function with nodes

[TABLE]

We designate weak convergence in $C(0,1)$ with uniform metrics by $\Longrightarrow$.

Theorem 1 *If $G$ exists, ${\rm det}\,G\neq 0$, then $Z_{n}\Longrightarrow~{}Z$. Here $Z$ is a centered Gaussian process with covariation function *

[TABLE]

Let $Z_{n}^{0}$ be an empirical bridge (see [28], [29], [30]):

[TABLE]

with $\widehat{\sigma}^{2}=\sum_{i=1}^{n}\widehat{\varepsilon}_{i}^{2}/n$. Let $L^{0}(t)=L(t)-tL(1)$.

Let $L_{n,j}$ be an empirical induced generalised Lorentz curve:

[TABLE]

$L_{n}=(L_{n,1},\ldots,L_{n,m})$, $L_{n}^{0}(t)=L_{n}(t)-tL_{n}(1)$.

Corollary 1 Let assumptions of Theorem 1 be held.

*1) Then $Z_{n}^{0}\Longrightarrow~{}Z^{0}$, a centered Gaussian process with covariation function *

[TABLE]

*2) Let $d\geq 1$ be integer, *

[TABLE]

$\widehat{g}_{ij}=\overline{X_{i}X_{j}}=\frac{1}{n}\sum_{k=1}^{n}X_{ki}X_{kj}$, $\widehat{G}=(\widehat{g}_{ij})_{i,j=1}^{m}$,

[TABLE]

$Q=(\widehat{K}^{0}(i/(d+1),j/(d+1)))_{i,j=1}^{d}$. Then ${\bf q}Q^{-1}{\bf q}^{T}$ converges weakly to a chi-squared distribution with $d$ degrees of freedom.

Note that ordering by $\xi_{i1}$, $i=1,\ldots,n$, can be viewed as ordering by $\delta_{i}$ with $h_{1}(x)=F^{-1}_{\xi_{11}}(x)$ (quantile function). In this case $L_{1}(t)=\int_{0}^{t}F^{-1}_{\xi_{11}}(x)\,dx$.

The next corollary is proved in [29].

Corollary 2 *Let $Y_{i}=\theta_{1}X_{i1}+\varepsilon_{i}$, $i=1,\ldots,n$, $\theta_{1}\in{\bf R}$, $(X_{11},\ldots,X_{n1})$ are order statistics of i.i.d. $(\xi_{11},\ldots,\xi_{n1})$, random variables $(\varepsilon_{1},\ldots,\varepsilon_{n})$ are i.i.d. and independent of them, $0<{\bf E}\,\xi_{11}^{2}<\infty$, ${\bf E}\,\varepsilon_{1}=0$, $0<{\bf Var}\,\varepsilon_{1}=\sigma^{2}<\infty$. Then $Z_{n}\Rightarrow Z$, a centered Gaussian process with covariance function *

[TABLE]

The next corollary is a partial case of Theorem 1 in [30].

Corollary 3 Let $Y_{i}=\theta_{1}X_{i1}+\theta_{2}+\varepsilon_{i}$, $i=1,\ldots,n$, $\theta_{1},\ \theta_{2}\in{\bf R}$, $(X_{11},\ldots,X_{n1})$ are order statistics of i.i.d. $(\xi_{11},\ldots,\xi_{n1})$, random variables $(\varepsilon_{1},\ldots,\varepsilon_{n})$ are i.i.d. and independent of them, $0<{\bf Var}\,\xi_{11}<\infty$, ${\bf E}\,\varepsilon_{1}=0$, $0<{\bf Var}\,\varepsilon_{1}=\sigma^{2}<\infty$. Then $Z_{n}\Rightarrow Z$, a centered Gaussian process with covariance function

[TABLE]

3 Proof of Theorem 1

Note that

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Note that $X_{[nt]}/n\to L(t)$ a.s. uniformely on compact sets, and ${X}^{T}{X}/n\to G$ a.s.

So we study process

[TABLE]

This process is a bounded linear functional of $(m+1)$-dimensional process

[TABLE]

We use the functional central limit theorem for induced order statistics by Davydov and Egorov [17].

We assume that $\eta_{i}=\xi_{i}\theta+e_{i}$, $\{e_{i}\}_{i=1}^{n}$ and $\{\xi_{i}\}_{i=1}^{n}$ are independent, $\{e_{i}\}_{i=1}^{n}$ are i.i.d., ${\bf E}\,e_{1}=0$, ${\bf Var}\,e_{1}=\sigma^{2}>0$.

Let see rows $(\delta_{i},\xi_{i}e_{i},e_{i})=(\delta_{i},\xi_{i1}e_{i},\ldots,\xi_{im}e_{i},e_{i})$. We have

[TABLE]

The conditional covariance matrix of the vector $(\xi_{1}e_{1},e_{1})$ is

[TABLE]

Let $\tilde{b}(x)$ be an upper triangular matrix such that $\tilde{b}(x)\tilde{b}(x)^{T}=\tilde{b}^{2}(x)$. Then

[TABLE]

Here $b(x)$ is an upper triangular matrix such that ${b}(x){b}(x)^{T}={b}^{2}(x)$. By Theorem 1 of Davydov and Egorov [17] the process

[TABLE]

converges weakly in the uniform metrics to the Gaussian process

[TABLE]

Here ${\bf W}_{m+1}=(W_{1},\ldots,W_{m+1})^{T}$ is an $(m+1)$-dimensional standard Wiener process.

So process

[TABLE]

converges weakly in the uniform metrics to the Gaussian process $Z=\{Z(t),\ t\in[0,1]\}$,

[TABLE]

${\bf W}_{m}=(W_{1},\ldots,W_{m})^{T}$.

By the noted convergencies $X_{[nt]}/n\to L(t)$ a.s. uniformely on compact sets, ${X}^{T}{X}/n\to G$ a.s., the process $Z_{n}$ has the same weak limit $Z$.

The covariance function of the limiting Gaussian process $Z$ is

[TABLE]

[TABLE]

[TABLE]

[TABLE]

The proof is complete.

Acknowledgement

The research was supported by RFBR grant 17-01-00683.