Preliminary test estimation in ULAN models

TL;DR

This paper develops a general asymptotic theory for preliminary test estimators within ULAN models, enabling detailed comparisons with classical estimators and illustrating results in both standard and complex modern regression setups.

Contribution

It establishes a comprehensive asymptotic framework for preliminary test estimators in ULAN models, including modern multisample covariance estimation scenarios.

Findings

Asymptotic behavior of preliminary test estimators is characterized in ULAN models.

Classical results are recovered in standard linear regression.

Simulation confirms theoretical predictions.

Abstract

Preliminary test estimation, which is a natural procedure when it is suspected a priori that the parameter to be estimated might take value in a submodel of the model at hand, is a classical topic in estimation theory. In the present paper, we establish general results on the asymptotic behavior of preliminary test estimators. More precisely, we show that, in uniformly locally asymptotically normal (ULAN) models, a general asymptotic theory can be derived for preliminary test estimators based on estimators admitting generic Bahadur-type representations. This allows for a detailed comparison between classical estimators and preliminary test estimators in ULAN models. Our results, that, in standard linear regression models, are shown to reduce to some classical results, are also illustrated in more modern and involved setups, such as the multisample one where $m$ covariance matrices ${\pmb\Sigma}_1, \ldots, {\pmb\Sigma}_m$ are to be estimated when it is suspected that these matrices might be equal, might be proportional, or might share a common "scale". Simulation results confirm our theoretical findings.

Equations

$${\hat{\mathbf{\theta }}}_{\mathrm{PTE}}:=\mathbb{I}[{\varphi }_n=1]{\hat{\mathbf{\theta }}}_{\mathrm{U}}+\mathbb{I}[{\varphi }_n=0]{\hat{\mathbf{\theta }}}_{\mathrm{C}},$$

$${\Lambda }^{(n)}:=\log \frac{{\mathrm{dP}}_{{\mathbf{\theta }}_n+{\mathbf{\nu }}_n{\mathbf{\tau }}_n}^{(n)}}{{\mathrm{dP}}_{{\mathbf{\theta }}_n}^{(n)}}={\mathbf{\tau }}_n^{\prime }{\mathbf{\Delta }}_{{\mathbf{\theta }}_n}^{(n)}-\frac{1}{2}{\mathbf{\tau }}_n^{\prime }{\mathbf{\Gamma }}_{\mathbf{\theta }}{\mathbf{\tau }}_n+o_{\mathrm{P}}(1)$$

$${\mathbf{\Theta }}_0=({\mathbf{\theta }}_0+\mathcal{M}(\mathbf{{\rm Y}}))\cap \mathbf{\Theta },$$

$${\hat{\mathbf{\theta }}}_{\mathrm{PTE}}=\mathbb{I}[{\varphi }_n=1]{\hat{\mathbf{\theta }}}_{\mathrm{U}}+\mathbb{I}[{\varphi }_n=0]{\hat{\mathbf{\theta }}}_{\mathrm{C}}$$

$$\Bigg{(}\begin{array}[]{cc}{\boldsymbol{\Sigma}}_{\boldsymbol{\theta}}&{\boldsymbol{\Omega}}_{\boldsymbol{\theta}}\\[-5.69054pt] {\boldsymbol{\Omega}}_{\boldsymbol{\theta}}&{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}\end{array}\Bigg{)}$$

$$\sqrt{n}({\hat{\mathbf{\theta }}}_{\mathrm{U}}-\mathbf{\theta })=\frac{1}{\sqrt{n}}\sum _{i=1}^n{\mathbf{T}}_i^{(n)}+o_{\mathrm{P}}(1),$$

$$\sqrt{n}({\hat{\mathbf{\theta }}}_{\mathrm{C}}-\mathbf{\theta })={\mathbf{P}}_{\mathbf{{\rm Y}}}\sqrt{n}({\hat{\mathbf{\theta }}}_{\mathrm{U}}-\mathbf{\theta })=\frac{1}{\sqrt{n}}\sum _{i=1}^n{\mathbf{P}}_{\mathbf{{\rm Y}}}{\mathbf{T}}_i^{(n)}+o_{\mathrm{P}}(1)$$

$$\sqrt{n}({\hat{\mathbf{\theta }}}_{\mathrm{U}}-\mathbf{\theta })={\mathbf{\Gamma }}_{\mathbf{\theta }}^{-1}{\mathbf{\Delta }}_{\mathbf{\theta }}^{(n)}+o_{\mathrm{P}}(1)$$

$$\sqrt{n}({\hat{\mathbf{\theta }}}_{\mathrm{C}}-\mathbf{\theta })=\mathbf{{\rm Y}}({\mathbf{{\rm Y}}}^{\prime }{\mathbf{\Gamma }}_{\mathbf{\theta }}\mathbf{{\rm Y}}{)}^{-1}{\mathbf{{\rm Y}}}^{\prime }{\mathbf{\Delta }}_{\mathbf{\theta }}^{(n)}+o_{\mathrm{P}}(1)$$

$$Q^{(n)}=\big{\|}({\bf I}_{p}-{\boldsymbol{\Gamma}}_{\hat{\boldsymbol{\theta}}_{\rm C}}^{1/2}{\boldsymbol{\Upsilon}}({\boldsymbol{\Upsilon}}^{\prime}{\boldsymbol{\Gamma}}_{\hat{\boldsymbol{\theta}}_{\rm C}}{\boldsymbol{\Upsilon}})^{-1}{\boldsymbol{\Upsilon}}^{\prime}{\boldsymbol{\Gamma}}_{\hat{\boldsymbol{\theta}}_{\rm C}}^{1/2}){\boldsymbol{\Gamma}}_{\hat{\boldsymbol{\theta}}_{\rm C}}^{-1/2}{\boldsymbol{\Delta}}_{\hat{\boldsymbol{\theta}}_{\rm C}}^{(n)}\big{\|}^{2}>\chi^{2}_{p-r,1-\alpha};$$

$$({\mathbf{I}}_p-{\mathbf{\Gamma }}_{{\hat{\mathbf{\theta }}}_{\mathrm{C}}}^{1/2}\mathbf{{\rm Y}}({\mathbf{{\rm Y}}}^{\prime }{\mathbf{\Gamma }}_{{\hat{\mathbf{\theta }}}_{\mathrm{C}}}\mathbf{{\rm Y}}{)}^{-1}{\mathbf{{\rm Y}}}^{\prime }{\mathbf{\Gamma }}_{{\hat{\mathbf{\theta }}}_{\mathrm{C}}}^{1/2}){\mathbf{\Gamma }}_{{\hat{\mathbf{\theta }}}_{\mathrm{C}}}^{-1/2}{\mathbf{\Delta }}_{{\hat{\mathbf{\theta }}}_{\mathrm{C}}}^{(n)}$$

$${\hat{\mathbf{\theta }}}_{\mathrm{PTE}}:=\mathbb{I}[{\varphi }_n=1]{\hat{\mathbf{\theta }}}_{\mathrm{U}}+\mathbb{I}[{\varphi }_n=0]{\hat{\mathbf{\theta }}}_{\mathrm{C}},$$

$${\hat{\mathbf{\theta }}}_{\mathrm{PTE}}:=(1-\lambda (Q^{(n)})){\hat{\mathbf{\theta }}}_{\mathrm{U}}+\lambda (Q^{(n)}){\hat{\mathbf{\theta }}}_{\mathrm{C}}.$$

$$\displaystyle{\boldsymbol{\mu}}_{\rm PTE}^{\rm Vic}=(1-\lambda(\|{\bf D}\|^{2}))\,\big{\{}(\mathbf{A}_{\boldsymbol{\theta}}{\boldsymbol{\Omega}}_{\boldsymbol{\theta}}-{\bf I}_{p}){\boldsymbol{\tau}}+{\bf A}_{\boldsymbol{\theta}}{\boldsymbol{\Sigma}}_{\boldsymbol{\theta}}{\bf C}_{\boldsymbol{\theta}}^{\prime}({\bf C}_{\boldsymbol{\theta}}{\boldsymbol{\Sigma}}_{\boldsymbol{\theta}}{\bf C}_{\boldsymbol{\theta}}^{\prime})^{-}({\bf D}-{\bf C}_{\boldsymbol{\theta}}{\boldsymbol{\Omega}}_{\boldsymbol{\theta}}{\boldsymbol{\tau}})\big{\}}$$

$${\mathbf{\Gamma }}_{\mathrm{PTE}}^{\mathrm{Vic}}=(1-\lambda (\Vert \mathbf{D}{\Vert }^2)){\mathbf{A}}_{\mathbf{\theta }}({\mathbf{\Sigma }}_{\mathbf{\theta }}-{\mathbf{L}}_{\mathbf{\theta }}){\mathbf{A}}_{\mathbf{\theta }}^{\prime }+\lambda (\Vert \mathbf{D}{\Vert }^2)\mathbf{{\rm Y}}{\mathbf{B}}_{\mathbf{\theta }}({\mathbf{\Sigma }}_{\mathbf{\theta }}-{\mathbf{L}}_{\mathbf{\theta }}){\mathbf{B}}_{\mathbf{\theta }}^{\prime }{\mathbf{{\rm Y}}}^{\prime },$$

$$\mathbf{z}\mapsto {\int }_{{\mathbb{R}}^p}{\varphi }_{{\mathbf{\mu }}_{\mathrm{PTE}}^{\mathrm{Vic}},{\mathbf{\Gamma }}_{\mathrm{PTE}}^{\mathrm{Vic}}}(\mathbf{z}){\varphi }_{{\mathbf{\mu }}_{\mathbf{D}},{\mathbf{\Sigma }}_{\mathbf{D}}}(\mathbf{x})d\mathbf{x},$$

$${\boldsymbol{\mu}}_{\rm PTE,eff}^{\rm Vic}={\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{-1/2}{\bf P}^{\perp}_{{\boldsymbol{\Upsilon}},{\rm eff}}\big{\{}(1-\lambda(\|{\bf D}\|^{2})){\bf D}-{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{1/2}{\boldsymbol{\tau}}\big{\}},$$

$${\mathbf{\Gamma }}_{\mathrm{PTE},\mathrm{eff}}^{\mathrm{Vic}}={\mathbf{\Gamma }}_{\mathbf{\theta }}^{-1/2}{\mathbf{P}}_{\mathbf{{\rm Y}},\mathrm{eff}}{\mathbf{\Gamma }}_{\mathbf{\theta }}^{-1/2},$$

$$\mathrm{E}[{\mathbf{\mu }}_{\mathrm{PTE},\mathrm{eff}}^{\mathrm{Vic}}]=-{\gamma }_2{\mathbf{\Gamma }}_{\mathbf{\theta }}^{-1/2}{\mathbf{P}}_{\mathbf{{\rm Y}},\mathrm{eff}}^{\perp }{\mathbf{\Gamma }}_{\mathbf{\theta }}^{1/2}\mathbf{\tau }$$

$$\mathrm{Var}[{\mathbf{\mu }}_{\mathrm{PTE},\mathrm{eff}}^{\mathrm{Vic}}]=(1-{\gamma }_2){\mathbf{\Gamma }}_{\mathbf{\theta }}^{-1/2}{\mathbf{P}}_{\mathbf{{\rm Y}},\mathrm{eff}}^{\perp }{\mathbf{\Gamma }}_{\mathbf{\theta }}^{-1/2}$$

$${\mathrm{AMSE}}_{\mathbf{\theta },\mathbf{\tau }}({\hat{\mathbf{\theta }}}_{\mathrm{PTE}}):=\mathrm{E}[\mathbf{Z}{\mathbf{Z}}^{\prime }]=\mathrm{Var}[\mathbf{Z}]+\mathrm{E}[\mathbf{Z}](\mathrm{E}[\mathbf{Z}]{)}^{\prime },$$

$${\mathrm{AMSE}}_{\mathbf{\theta },\mathbf{\tau }}({\hat{\mathbf{\theta }}}_{\mathrm{PTE}})$$

$${\mathrm{AMSE}}_{\mathbf{\theta },\mathbf{\tau }}({\hat{\mathbf{\theta }}}_{\mathrm{U}})={\mathbf{\Gamma }}_{\mathbf{\theta }}^{-1}$$

$${\mathrm{AMSE}}_{\mathbf{\theta },\mathbf{\tau }}({\hat{\mathbf{\theta }}}_{\mathrm{C}})={\mathbf{\Gamma }}_{\mathbf{\theta }}^{-1/2}{\mathbf{P}}_{\mathbf{{\rm Y}},\mathrm{eff}}{\mathbf{\Gamma }}_{\mathbf{\theta }}^{-1/2}+{\mathbf{\Gamma }}_{\mathbf{\theta }}^{-1/2}{\mathbf{P}}_{\mathbf{{\rm Y}},\mathrm{eff}}^{\perp }{\mathbf{\Gamma }}_{\mathbf{\theta }}^{1/2}\mathbf{\tau }{\mathbf{\tau }}^{\prime }{\mathbf{\Gamma }}_{\mathbf{\theta }}^{1/2}{\mathbf{P}}_{\mathbf{{\rm Y}},\mathrm{eff}}^{\perp }{\mathbf{\Gamma }}_{\mathbf{\theta }}^{-1/2},$$

$${\mathrm{AMSE}}_{\mathbf{\theta },\mathbf{\tau }}({\hat{\mathbf{\theta }}}_{\mathrm{PTE}})$$

$${\mathrm{AMSE}}_{\mathbf{\theta },\mathbf{\tau }}^{\mathrm{s}}(\hat{\mathbf{\theta }}):=\mathrm{tr}[{\mathbf{\Gamma }}_{\mathbf{\theta }}^{1/2}({\mathrm{AMSE}}_{\mathbf{\theta },\mathbf{\tau }}(\hat{\mathbf{\theta }})){\mathbf{\Gamma }}_{\mathbf{\theta }}^{1/2}],$$

$${\mathrm{AMSE}}_{\mathbf{\theta },\mathbf{\tau }}^{\mathrm{s}}({\hat{\mathbf{\theta }}}_{\mathrm{C}})=\mathrm{tr}[{\mathbf{P}}_{\mathbf{{\rm Y}},\mathrm{eff}}]+\mathrm{tr}[{\mathbf{P}}_{\mathbf{{\rm Y}},\mathrm{eff}}^{\perp }{\mathbf{\Gamma }}_{\mathbf{\theta }}^{1/2}\mathbf{\tau }{\mathbf{\tau }}^{\prime }{\mathbf{\Gamma }}_{\mathbf{\theta }}^{1/2}{\mathbf{P}}_{\mathbf{{\rm Y}},\mathrm{eff}}^{\perp }]=r+\Vert \mathbf{\delta }{\Vert }^2,$$

$${\mathrm{AMSE}}_{\mathbf{\theta },\mathbf{\tau }}^{\mathrm{s}}({\hat{\mathbf{\theta }}}_{\mathrm{PTE}})=p-{\gamma }_2(p-r)+(2{\gamma }_2-{\gamma }_4)\Vert \mathbf{\delta }{\Vert }^2,$$

$$\mathbf{Y}=\rho 1_n+\beta \mathbf{x}+\mathbf{ϵ},$$

$${\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}=\frac{1}{\sigma^{2}}\Bigg{(}\begin{array}[]{cc}1&\bar{x}_{0}\\[-5.69054pt] \bar{x}_{0}&s_{0}+\bar{x}_{0}^{2}\end{array}\Bigg{)},$$

Full text

We now move to cases (i)–(ii), where we will actually consider parameter sequences of the form ${\boldsymbol{\theta}}_{n}={\boldsymbol{\theta}}+{\boldsymbol{\nu}}_{n}{\boldsymbol{\tau}}_{n}\in{\boldsymbol{\Theta}}$, with ${\boldsymbol{\theta}}\in{\boldsymbol{\Theta}}_{0}$ and $({\boldsymbol{\tau}}_{n})\to{\boldsymbol{\tau}}$ (note that case (i) is obtained for ${\boldsymbol{\tau}}_{n}\equiv\mathbf{0}$). We have the following result (see the appendix for a proof).

[TABLE]

and covariance matrix

[TABLE]

where we denoted as ${\bf A}^{-}$ the Moore-Penrose inverse of ${\bf A}$ and where we let ${\bf L}_{\boldsymbol{\theta}}:={\boldsymbol{\Sigma}}_{\boldsymbol{\theta}}{\bf C}_{\boldsymbol{\theta}}^{\prime}({\bf C}_{\boldsymbol{\theta}}{\boldsymbol{\Sigma}}_{\boldsymbol{\theta}}{\bf C}_{\boldsymbol{\theta}}^{\prime})^{-}{\bf C}_{\boldsymbol{\theta}}{\boldsymbol{\Sigma}}_{\boldsymbol{\theta}}$.

Theorem 2 allows us to obtain an expression for the unconditional asymptotic distribution of ${\boldsymbol{\nu}}_{n}^{-1}(\hat{\boldsymbol{\theta}}_{\rm PTE}-{\boldsymbol{\theta}}_{n})$: in the framework of Assumption (B), the Le Cam third lemma implies that $\mathbf{D}_{n}$ is asymptotically normal with mean vector ${\boldsymbol{\mu}}_{\mathbf{D}}:={\bf C}_{\boldsymbol{\theta}}{\boldsymbol{\Omega}}_{\boldsymbol{\theta}}{\boldsymbol{\tau}}$ and covariance matrix ${\boldsymbol{\Sigma}}_{\mathbf{D}}={\bf C}_{\boldsymbol{\theta}}{\boldsymbol{\Sigma}}_{\boldsymbol{\theta}}{\bf C}_{\boldsymbol{\theta}}^{\prime}$ under ${\rm P}^{(n)}_{{\boldsymbol{\theta}}_{n}}$, so that, under the same sequence of hypotheses, ${\boldsymbol{\nu}}_{n}^{-1}(\hat{\boldsymbol{\theta}}_{\rm PTE}-{\boldsymbol{\theta}}_{n})$ converges weakly to a random $p$-vector ${\bf Z}$ with probability density function (pdf)

[TABLE]

where $\phi_{{\boldsymbol{\mu}},{\boldsymbol{\Sigma}}}$ stands for the pdf of the $p$-variate normal distribution with mean vector ${\boldsymbol{\mu}}$ and covariance matrix ${\boldsymbol{\Sigma}}$. Since the pdf (3.9) does not allow for a simple comparison between $\hat{\boldsymbol{\theta}}_{\rm PTE}$, $\hat{\boldsymbol{\theta}}_{\rm U}$ and $\hat{\boldsymbol{\theta}}_{\rm C}$, we will base such a comparison on the asymptotic mean square errors (MSEs) of these estimators.

A general expression for the asymptotic MSEs can be obtained by computing ${\rm E}[{\boldsymbol{\mu}}_{\rm PTE}^{\rm Vic}]$, ${\rm Var}[{\boldsymbol{\mu}}_{\rm PTE}^{\rm Vic}]$ and ${\rm E}[{\boldsymbol{\Gamma}}_{\rm PTE}^{\rm Vic}]$, recalling that, under ${\rm P}^{(n)}_{{\boldsymbol{\theta}}_{n}}$, the random vector $\mathbf{D}^{(n)}$ has asymptotic mean ${\boldsymbol{\mu}}_{\mathbf{D}}$ and covariance matrix ${\boldsymbol{\Sigma}}_{\mathbf{D}}$. We now derive these limiting MSEs when PTEs are based on $Q^{(n)}$ in (2.5) and on asymptotically efficient estimators satisfying (2.3)–(2.4) (limiting MSEs of PTEs based on other estimators can be obtained in the same way). For such estimators and tests, Theorem 2 yields that, conditional on ${\bf D}^{(n)}={\bf D}$, ${\boldsymbol{\nu}}_{n}^{-1}(\hat{\boldsymbol{\theta}}_{\rm PTE}-{\boldsymbol{\theta}}_{n})$ is, under ${\rm P}^{(n)}_{{\boldsymbol{\theta}}_{n}}$, asymptotically normal with mean vector

[TABLE]

and covariance matrix

[TABLE]

with ${\bf P}_{{\boldsymbol{\Upsilon}},{\rm eff}}:={\boldsymbol{\Gamma}}_{{\boldsymbol{\theta}}}^{1/2}{\boldsymbol{\Upsilon}}({\boldsymbol{\Upsilon}}^{\prime}{\boldsymbol{\Gamma}}_{{\boldsymbol{\theta}}}{\boldsymbol{\Upsilon}})^{-1}{\boldsymbol{\Upsilon}}^{\prime}{\boldsymbol{\Gamma}}_{{\boldsymbol{\theta}}}^{1/2}$ and ${\bf P}^{\perp}_{{\boldsymbol{\Upsilon}},{\rm eff}}:={\bf I}_{p}-{\bf P}_{{\boldsymbol{\Upsilon}},{\rm eff}}$. We then have the following result (see the appendix for a proof).

[TABLE]

and

[TABLE]

where we let $\gamma_{j}:={\rm P}[V_{j}\leq\chi^{2}_{p-r,1-\alpha}]$, with $V_{j}\sim\chi^{2}_{p-r+j}({\boldsymbol{\tau}}^{\prime}{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{1/2}{\bf P}_{{\boldsymbol{\Upsilon}},{\rm eff}}^{\perp}{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{1/2}{\boldsymbol{\tau}})$ $($throughout, $\chi^{2}_{\ell}(\eta)$ will stand for the non-central chi-square distribution with $\ell$ degrees of freedom and with non-centrality parameter $\eta)$.

We define the limiting MSE of $\hat{\boldsymbol{\theta}}_{\rm PTE}$ under ${\rm P}^{(n)}_{{\boldsymbol{\theta}}_{n}}$ as

[TABLE]

where ${\bf Z}$ is the weak limit of ${\boldsymbol{\nu}}_{n}^{-1}(\hat{\boldsymbol{\theta}}_{\rm PTE}-{\boldsymbol{\theta}}_{n})$ under ${\rm P}^{(n)}_{{\boldsymbol{\theta}}_{n}}$. Now, since ${\rm E}[\mathbf{Z}]={\rm E}[{\rm E}[\mathbf{Z}|{\bf D}]]={\rm E}[{\boldsymbol{\mu}}_{\rm PTE,eff}^{\rm Vic}]$ and ${\rm Var}[\mathbf{Z}]={\rm E}[{\rm Var}[\mathbf{Z}|{\bf D}]]+{\rm Var}[{\rm E}[\mathbf{Z}|{\bf D}]]={\boldsymbol{\Gamma}}_{\rm PTE,eff}^{\rm Vic}+{\rm Var}[{\boldsymbol{\mu}}_{\rm PTE,eff}^{\rm Vic}]$ (note that ${\rm Var}[\mathbf{Z}|{\bf D}]={\boldsymbol{\Gamma}}_{\rm PTE,eff}^{\rm Vic}$ is non-random), Proposition 1 yields

[TABLE]

To enable proper comparison with the unconstrained and constrained antecedents of $\hat{\boldsymbol{\theta}}_{\rm PTE}$ (namely, the estimators $\hat{\boldsymbol{\theta}}_{\rm U}$ and $\hat{\boldsymbol{\theta}}_{\rm C}$ satisfying (2.3) and (2.4), respectively), the following result provides explicit expressions for the limiting MSEs of these estimators (see the appendix for a proof).

[TABLE]

and

[TABLE]

where $\hat{\boldsymbol{\theta}}_{\rm U}$ and $\hat{\boldsymbol{\theta}}_{\rm C}$ are estimators satisfying (2.3) and (2.4), respectively.

It is worthwile to consider some boundary cases. For $\alpha=1$, we have $\gamma_{2}=\gamma_{4}=0$, so that ${\rm AMSE}_{{\boldsymbol{\theta}},{\boldsymbol{\tau}}}(\hat{\boldsymbol{\theta}}_{\rm PTE})={\rm AMSE}_{{\boldsymbol{\theta}},{\boldsymbol{\tau}}}(\hat{\boldsymbol{\theta}}_{\rm U})$, which is compatible with the fact that $\hat{\boldsymbol{\theta}}_{\rm PTE}=\hat{\boldsymbol{\theta}}_{\rm U}$ almost surely when the test $\phi_{n}$ is performed at asymptotic level $\alpha=1$. At the other extreme, for $\alpha=0$, we rather have $\gamma_{2}=\gamma_{4}=1$, which provides

[TABLE]

in agreement with the fact that $\hat{\boldsymbol{\theta}}_{\rm PTE}=\hat{\boldsymbol{\theta}}_{\rm C}$ almost surely when the test $\phi_{n}$ is performed at asymptotic level $\alpha=0$.

To conclude this section, we offer a comparison between ${\rm AMSE}_{{\boldsymbol{\theta}},{\boldsymbol{\tau}}}(\hat{\boldsymbol{\theta}}_{\rm PTE})$, ${\rm AMSE}_{{\boldsymbol{\theta}},{\boldsymbol{\tau}}}(\hat{\boldsymbol{\theta}}_{\rm U})$, and ${\rm AMSE}_{{\boldsymbol{\theta}},{\boldsymbol{\tau}}}(\hat{\boldsymbol{\theta}}_{\rm C})$. These limiting MSEs being matrix-valued, it is needed to base this comparison on a scalar summary, such as, e.g., their trace. In the present case, where the unconstrained estimator satisfies ${\rm AMSE}_{{\boldsymbol{\theta}},{\boldsymbol{\tau}}}(\hat{\boldsymbol{\theta}}_{\rm U})={\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{-1}$, it is natural to measure the asymptotic performance of an estimator $\hat{{\boldsymbol{\theta}}}$ through the equivalent scalar quantity

[TABLE]

which, for $\hat{\boldsymbol{\theta}}_{\rm U}$, will provide the “normalized” perfomance ${\rm AMSE}^{\rm s}_{{\boldsymbol{\theta}},{\boldsymbol{\tau}}}(\hat{\boldsymbol{\theta}}_{\rm U})=p$ (see Proposition 2), that does not depend on the value of ${\boldsymbol{\theta}}$ at which the contiguous alternatives ${\boldsymbol{\theta}}_{n}={\boldsymbol{\theta}}+{\boldsymbol{\nu}}_{n}{\boldsymbol{\tau}}_{n}$ are localized. Proposition 2 also entails that

[TABLE]

with ${\boldsymbol{\delta}}:={\bf P}_{{\boldsymbol{\Upsilon}},{\rm eff}}^{\perp}{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{1/2}{\boldsymbol{\tau}}$. Note that, at ${\boldsymbol{\tau}}=0$, this shows that ${\rm AMSE}^{\rm s}_{{\boldsymbol{\theta}},{\boldsymbol{\tau}}}(\hat{\boldsymbol{\theta}}_{\rm C})=r<p={\rm AMSE}^{\rm s}_{{\boldsymbol{\theta}},{\boldsymbol{\tau}}}(\hat{\boldsymbol{\theta}}_{\rm U})$, which confirms the intuition that $\hat{{\boldsymbol{\theta}}}_{\rm C}$ dominates $\hat{{\boldsymbol{\theta}}}_{\rm U}$ when the true parameter value belongs to ${\boldsymbol{\Theta}}_{0}$. Now, it easily follows from (3.12) that

[TABLE]

where $\gamma_{j}={\rm P}[V_{j}\leq\chi^{2}_{p-r,1-\alpha}]$, with $V_{j}\sim\chi^{2}_{p-r+j}(\|{\boldsymbol{\delta}}\|^{2})$. Figure 2 plots, for $p=10$, $r=1$ and $\alpha=.05$, the quantities ${\rm AMSE}^{\rm s}_{{\boldsymbol{\theta}},{\boldsymbol{\tau}}}(\hat{\boldsymbol{\theta}}_{\rm U})$, ${\rm AMSE}^{\rm s}_{{\boldsymbol{\theta}},{\boldsymbol{\tau}}}(\hat{\boldsymbol{\theta}}_{\rm C})$ and ${\rm AMSE}^{\rm s}_{{\boldsymbol{\theta}},{\boldsymbol{\tau}}}(\hat{\boldsymbol{\theta}}_{\rm PTE})$ as functions of $\|{\boldsymbol{\delta}}\|^{2}$. The figure reveals that, under ${\rm P}_{\boldsymbol{\theta}}^{(n)}$ with ${\boldsymbol{\theta}}\in{\boldsymbol{\Theta}}_{0}$ (which corresponds to ${\boldsymbol{\delta}}=0$), the constrained estimator $\hat{{\boldsymbol{\theta}}}_{\rm C}$ has the best performance, as expected. The PTE performs better than $\hat{{\boldsymbol{\theta}}}_{\rm U}$ in the vicinity of the constraint ($\|{\boldsymbol{\delta}}\|$ small to moderate) and it is asymptotically equivalent to $\hat{{\boldsymbol{\theta}}}_{\rm U}$ far from the constraint ($\|{\boldsymbol{\delta}}\|$ large).

4. Two specific applications

In this section, we illustrate the general results obtained above on two particular cases. First, we consider preliminary test estimation in the simple linear regression model and show that we recover for this model and for the considered estimation problem the classical results of Saleh (2006) (Section 4.1). Then, we consider the joint estimation of $m$ covariance matrices ${\boldsymbol{\Sigma}}_{1},\ldots,{\boldsymbol{\Sigma}}_{m}$ in a context where it is suspected that these covariance matrices might be equal, might be proportional, or might share a common “scale” (Section 4.2).

4.1 Simple linear regression

Consider the simple linear regression model

[TABLE]

where $\mathbf{Y}=(Y_{1},\ldots,Y_{n})^{\prime}$ is a response vector, $\mathbf{x}=(x_{1},\ldots,x_{n})^{\prime}$ is a vector of non-random covariates, and where the error vector ${\boldsymbol{\epsilon}}=(\epsilon_{1},\ldots,\epsilon_{n})^{\prime}$ is multinormal with mean zero and covariance matrix $\sigma^{2}{\bf I}_{n}$, for some $\sigma^{2}>0$. This is the classical simple linear model with intercept $\rho$, slope $\beta$, and Gaussian homoscedastic errors with variance $\sigma^{2}$. Throughout, we consider the parameter ${\boldsymbol{\theta}}:=(\rho,\beta)^{\prime}$, as we will assume that $\sigma^{2}$ is known (this is actually no restriction, since the block-diagonality of the Fisher information matrix in this model entails that replacing $\sigma^{2}$ with a root-$n$ consistent estimator will have no asymptotic cost, so that all results we obtain below extend to the case where $\sigma^{2}$ would remain an unspecified nuisance). One can easily show that this model is ULAN, with a central sequence ${\boldsymbol{\Delta}}_{\boldsymbol{\theta}}^{(n)}$ that, under ${\rm P}_{\boldsymbol{\theta}}^{(n)}$, is asymptotically normal with mean zero and covariance matrix

[TABLE]

where $\bar{x}_{0}:=\lim_{n\rightarrow\infty}n^{-1}\sum_{i=1}^{n}x_{i}$ and $s_{0}:=\lim_{n\rightarrow\infty}s_{x}^{(n)}$ with $s_{x}^{(n)}:=n^{-1}\mathbf{x}^{\prime}\mathbf{x}-n^{-2}({\bf 1}_{n}^{\prime}\mathbf{x})^{2}$; of course, we tacitly assume that these limits exist and are finite. We consider here preliminary test estimation of ${\boldsymbol{\theta}}$ when it is suspected that $\beta=\beta_{0}$ for some given $\beta_{0}$. In the context, the classical, unconstrained, estimator of ${\boldsymbol{\theta}}$ is the maximum likelihood estimator

[TABLE]

whereas the natural constrained estimator would be $\hat{{\boldsymbol{\theta}}}_{\rm C}:={\tilde{\rho}\choose\beta_{0}}$, with $\tilde{\rho}:=n^{-1}({\bf 1}_{n}^{\prime}\mathbf{Y}-{\beta}_{0}{\bf 1}_{n}^{\prime}\mathbf{x})$. Since the locally asymptotically optimal test for $\mathcal{H}_{0}:\beta=\beta_{0}$ against $\mathcal{H}_{1}:\beta\neq\beta_{0}$ rejects the null hypothesis at asymptotic level $\alpha$ when

[TABLE]

the resulting PTE is given by

[TABLE]

Letting ${\boldsymbol{\theta}}_{0}={\rho\choose\beta_{0}}$ be an arbitrary value of the parameter of interest corresponding to the constraint, the null hypothesis can be written as ${\cal H}_{0}:{\boldsymbol{\theta}}\in{{\boldsymbol{\theta}}_{0}}+{\cal M}({\boldsymbol{\Upsilon}})$, with ${\boldsymbol{\Upsilon}}:={1\choose 0}$. Since

[TABLE]

it follows from (3.12) that, under ${\rm P}^{(n)}_{{\boldsymbol{\theta}}_{0}+n^{-1/2}{\boldsymbol{\tau}}}$, with ${\boldsymbol{\tau}}={0\choose\delta}$, the MSE quantity ${\rm AMSE}_{{\boldsymbol{\theta}},{\boldsymbol{\tau}}}(\hat{{\boldsymbol{\theta}}}_{\rm PTE})$ is here given by

[TABLE]

where the $\gamma_{j}$’s are computed with $p=2$ and $r=1$. This is in perfect agreement with the result in Theorem 4, p.p. 94–96 in Saleh (2006).

4.2 Multisample estimation of covariance matrices

Consider $m(\geq 2)$ mutually independent samples of random $k$-vectors $\mathbf{X}_{i1},\ldots,\mathbf{X}_{in_{i}}$, $i=1,\ldots,m$, with respective sample sizes $n_{1},\ldots,n_{m}$, such that, for any $i$, the $\mathbf{X}_{ij}$’s form a random sample from the multinormal distribution with mean vector ${\bf 0}$ and (invertible) covariance matrix ${\boldsymbol{\Sigma}}_{i}$ (all results below extend to the case where observations in the $i$th sample would have a common, unspecified, mean ${\boldsymbol{\mu}}_{i}$, $i=1,\ldots,n$, due to the block-diagonality of the Fisher information matrix for location and scatter in elliptical models; see, e.g., Hallin and Paindaveine, 2006). In the sequel, we decompose the covariance matrices into ${\boldsymbol{\Sigma}}_{i}=\sigma^{2}_{i}\mathbf{V}_{i}$, where $\sigma_{i}^{2}:=({\rm det}\,{\boldsymbol{\Sigma}}_{i})^{1/k}$ is their “scale” and ${\mathbf{V}}_{i}:={\boldsymbol{\Sigma}}_{i}/({\rm det}\,{\boldsymbol{\Sigma}}_{i})^{1/k}$ is their “shape”. Under the only assumption that $\lambda_{i}:=\lambda_{i}^{(n)}:=n_{i}/n:=n_{i}/(\sum_{\ell=1}^{m}n_{\ell})\to\lambda_{i}\in(0,1)$ (to make the notation lighter, we will not stress the dependence in $n$ in many quantities below), it follows from Hallin and Paindaveine (2009) that the sequence of Gaussian models indexed by

[TABLE]

where ${\rm ve}\!^{{}^{{\scriptscriptstyle\circ}}}\!\!{\rm ch}\,\mathbf{V}(\in\mathbb{R}^{d_{k}}$, with $d_{k}:=k(k+1)/2-1)$ stands for the vector obtained by depriving ${\rm vech}\mathbf{V}$ of its first entry $\mathbf{V}_{11}$, is ULAN in the sense of Assumption A. To describe the corresponding central sequence and Fisher information matrix, we need the following notation.

Denoting as ${\bf e}_{r}$ the $r$th vector of the canonical basis of $\mathbb{R}^{k}$, we let ${\bf K}_{k}:=\sum_{r,s=1}^{k}({\bf e}_{r}{\bf e}_{s}^{\prime})\otimes({\bf e}_{s}{\bf e}_{r}^{\prime})$ be the $k^{2}\times k^{2}$ commutation matrix, put ${\bf J}_{k}:=({\rm vec}\,{\bf I}_{k})({\rm vec}\,{\bf I}_{k})^{\prime}$, and define ${\bf M}_{k}({\bf V})$ as the $(d_{k}\times k^{2})$ matrix such that $({\bf M}_{k}({\bf V}))^{\prime}({\rm ve}\!^{{}^{{\scriptscriptstyle\circ}}}\!\!{\rm ch}\,{\bf v})={\rm vec}\,{\bf v}$ for any symmetric $k\times k$ matrix ${\bf v}$ such that ${\rm tr}[{\bf V}^{-1}{\bf v}]=0$. We further put

[TABLE]

Then, letting $\mathbf{S}_{i}:=n_{i}^{-1}\sum_{j=1}^{n_{i}}\mathbf{X}_{ij}\mathbf{X}_{ij}^{\prime}$ be the empirical covariance matrix in sample $i$, the central sequence is ${\boldsymbol{\Delta}}_{\boldsymbol{\theta}}=(\Delta_{\boldsymbol{\theta}}^{{\scriptscriptstyle I},1},\ldots,\Delta_{\boldsymbol{\theta}}^{{\scriptscriptstyle I},m},{\boldsymbol{\Delta}}_{\boldsymbol{\theta}}^{{\scriptscriptstyle I\!I},1},\ldots,{\boldsymbol{\Delta}}_{\boldsymbol{\theta}}^{{\scriptscriptstyle I\!I},m})$, where, for $i=1,\ldots,m$, we wrote

[TABLE]

whereas the (full-rank and block-diagonal) information matrix takes the form ${\boldsymbol{\Gamma}}_{{\boldsymbol{\theta}}}:={\rm diag}({\boldsymbol{\Gamma}}^{{\scriptscriptstyle I}}_{{\boldsymbol{\theta}}},{\boldsymbol{\Gamma}}^{{\scriptscriptstyle I\!I}}_{{\boldsymbol{\theta}}}):={\rm diag}(\frac{k}{2}\underline{\boldsymbol{\sigma}}^{-4},{\bf H}_{k}(\underline{\bf V}))$, with $\underline{\boldsymbol{\sigma}}:={\rm diag}(\sigma_{1},\ldots,\sigma_{m})$ and ${\bf H}_{k}(\underline{\bf V}):={\rm diag}({\bf H}_{k}({\bf V}_{1}),\ldots,{\bf H}_{k}({\bf V}_{m}))$. The corresponding contiguity rate ${\boldsymbol{\nu}}_{n}$ in Assumption A is given by ${\boldsymbol{\nu}}_{n}=n^{-1/2}{\bf r}_{n}$, where

[TABLE]

We consider here estimation of ${\boldsymbol{\Sigma}}_{1},\ldots,{\boldsymbol{\Sigma}}_{m}$ or, equivalently, estimation of ${\boldsymbol{\theta}}$. An advantage of the ${\boldsymbol{\theta}}$-parametrization is that it allows the construction of various PTEs: one may suspect, e.g., scale homogeneity ${\cal H}_{0}^{\rm scale}:\sigma^{2}_{1}=\ldots=\sigma^{2}_{m}$, shape homogeneity ${\cal H}_{0}^{\rm shape}:\mathbf{V}_{1}=\ldots=\mathbf{V}_{m}$, or full covariance homogeneity ${\cal H}_{0}^{\rm cov}:\sigma^{2}_{1}\mathbf{V}_{1}=\ldots=\sigma_{m}^{2}\mathbf{V}_{m}$, that is, ${\cal H}_{0}^{\rm cov}:{\boldsymbol{\Sigma}}_{1}=\ldots={\boldsymbol{\Sigma}}_{m}$. An asymptotically efficient unconstrained estimator in this Gaussian model is given by

[TABLE]

whereas, writing $\mathbf{S}:=n^{-1}\sum_{i=1}^{m}\sum_{j=1}^{n_{i}}\mathbf{X}_{ij}\mathbf{X}_{ij}^{\prime}$ for the pooled covariance matrix estimator, asymptotically efficient constrained estimators, for the three constraints ${\cal H}_{0}^{\rm scale}$, ${\cal H}_{0}^{\rm shape}$ and ${\cal H}_{0}^{\rm cov}$ above, are given by

[TABLE]

[TABLE]

and

[TABLE]

respectively. The three hypotheses ${\cal H}_{0}^{\rm scale}$, ${\cal H}_{0}^{\rm shape}$ and ${\cal H}_{0}^{\rm cov}$ impose linear restrictions on ${\boldsymbol{\theta}}$, hence can be written as

[TABLE]

(more specifically, ${\boldsymbol{\Upsilon}}_{\rm scale}:={\rm diag}({\bf 1}_{m},{\bf I}_{md_{k}})$, ${\boldsymbol{\Upsilon}}_{\rm shape}:={\rm diag}({\bf I}_{m},{\bf 1}_{m}\otimes{\bf I}_{d_{k}})$ and ${\boldsymbol{\Upsilon}}_{\rm cov}:={\rm diag}({\bf 1}_{m},{\bf 1}_{m}\otimes{\bf I}_{d_{k}})$). Now, if the $p\times r$ matrix ${\boldsymbol{\Upsilon}}$ stands for either of ${\boldsymbol{\Upsilon}}_{\rm scale}$, ${\boldsymbol{\Upsilon}}_{\rm shape}$ or ${\boldsymbol{\Upsilon}}_{\rm cov}$ (of course, each constraint matrix has its own $r$), the locally asymptotically most stringent test $\phi^{(n)}_{{\boldsymbol{\Upsilon}}}$ for ${\cal H}_{0}:{\boldsymbol{\theta}}\in{\cal M}({\boldsymbol{\Upsilon}})$ rejects the null hypothesis at asymptotic level $\alpha$ when

[TABLE]

This allows us to consider the PTEs

[TABLE]

[TABLE]

and

[TABLE]

To compare these PTEs with their unconstrained and constrained antecedents, we performed the following Monte Carlo exercise, that focuses on the case $m=2$, $k=2$ and $n_{1}=n_{2}=20,\!000$. We generated independently $M=10,\!000$ samples of mutually independent observations $(\mathbf{X}_{1},\ldots,\mathbf{X}_{n1},{\mathbf{Y}}_{1}(\ell),\ldots,{\mathbf{Y}}_{n_{2}}(\ell))$, $\ell=0,\ldots,9$, where the $\mathbf{X}_{i}$’s are ${\cal N}({\bf 0},{\bf I}_{k})$ and the $\mathbf{Y}_{i,\ell}$’s are ${\cal N}({\bf 0},{\boldsymbol{\Sigma}}_{\ell})$, with

[TABLE]

For $\ell=0$, the samples $\mathbf{X}_{1},\ldots,\mathbf{X}_{n_{1}}$ and $\mathbf{Y}_{1}(\ell),\ldots,\mathbf{Y}_{n}(\ell)$ share the same underlying covariance matrix ${\bf I}_{p}$, hence also the same scales and shapes, whereas $\ell=1,\ldots,9$ provide increasingly distinct scales and shapes. In other words, the constraints above are met for $\ell=0$ and are increasingly violated for $\ell=1,\ldots,9$. For every considered estimator $\hat{{\boldsymbol{\theta}}}$ of the resulting true parameter value ${\boldsymbol{\theta}}$, we measure the performance of $\hat{{\boldsymbol{\theta}}}$ through

[TABLE]

where $\hat{{\boldsymbol{\theta}}}^{(m)}$ is an estimator computed in the $m$th replication ($m=1,\ldots,M$), or rather, parallel to what we did in Section 3, through the scalar quantity $\hat{\rm AMSE}^{\rm s}_{\boldsymbol{\theta}}(\hat{{\boldsymbol{\theta}}}):={\rm tr}[{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}\hat{\rm AMSE}_{\boldsymbol{\theta}}(\hat{{\boldsymbol{\theta}}})]$. Figure 3 then plots $\hat{\rm AMSE}^{\rm s}(\hat{{\boldsymbol{\theta}}})$ for the PTEs $\hat{{\boldsymbol{\theta}}}_{\rm PTE}^{\rm scale}$, $\hat{{\boldsymbol{\theta}}}_{\rm PTE}^{\rm shape}$ and $\hat{{\boldsymbol{\theta}}}_{\rm PTE}^{\rm cov}$ (the corresponding tests are all performed at asymptotic level $\alpha=.05$), as well as their constrained and unconstrained antecedents ${\hat{\boldsymbol{\theta}}}_{\rm C}^{\rm scale}$, ${\hat{\boldsymbol{\theta}}}_{\rm C}^{\rm shape}$, ${\hat{\boldsymbol{\theta}}}_{\rm C}^{\rm cov}$ and ${\hat{\boldsymbol{\theta}}}_{\rm U}$. To match what was done in Figure 2, these quantities are not plotted as functions of $\ell$, but rather as functions of the induced $\|{\boldsymbol{\delta}}\|^{2}$. The figure also provides the corresponding asymptotic performance measures ${\rm AMSE}^{\rm s}_{{\boldsymbol{\theta}},{\boldsymbol{\tau}}}(\hat{{\boldsymbol{\theta}}})$ resulting from the general expression obtained in Section 3. Clearly, the results show that that these empirical and theoretical performance measures are in a perfect agreement.

Acknowledgements

Davy Paindaveine’s research is supported by a research fellowship from the Francqui Foundation and by the Program of Concerted Research Actions (ARC) of the Université libre de Bruxelles. Thomas Verdebout’s research is supported by the ARC Program of the Université libre de Bruxelles and by the Crédit de Recherche J.0134.18 of the FNRS (Fonds National pour la Recherche Scientifique), Communauté Française de Belgique.

Appendix: Proofs

In this appendix, we collect the proofs of the various results.

Proof of Theorem 1. First note that

[TABLE]

For any $\varepsilon>0$, Assumption A(iii) ensures that

[TABLE]

so that $\lambda(Q^{(n)}){\boldsymbol{\nu}}_{n}^{-1}=o_{\rm P}(1)$ under ${\rm P}^{(n)}_{\boldsymbol{\theta}}$. Since by assumption, ${\hat{\boldsymbol{\theta}}}_{\rm C}-{\hat{\boldsymbol{\theta}}}_{\rm U}={\hat{\boldsymbol{\theta}}}_{\rm C}-{{\boldsymbol{\theta}}}+o_{\rm P}(1)=O_{\rm P}(1)$ under ${\rm P}^{(n)}_{\boldsymbol{\theta}}$, the result follows from (E.19). $\square$

Proof of Theorem 2. Writing ${\boldsymbol{\nu}}_{n}^{-1}(\hat{\boldsymbol{\theta}}_{\rm U}-{\boldsymbol{\theta}}_{n})={\boldsymbol{\nu}}_{n}^{-1}(\hat{\boldsymbol{\theta}}_{\rm U}-{\boldsymbol{\theta}})-{\boldsymbol{\nu}}_{n}^{-1}({\boldsymbol{\theta}}_{n}-{\boldsymbol{\theta}})$ and ${\boldsymbol{\nu}}_{n}^{-1}(\hat{\boldsymbol{\theta}}_{\rm C}-{\boldsymbol{\theta}}_{n})={\boldsymbol{\nu}}_{n}^{-1}(\hat{\boldsymbol{\theta}}_{\rm C}-{\boldsymbol{\theta}})-{\boldsymbol{\nu}}_{n}^{-1}({\boldsymbol{\theta}}_{n}-{\boldsymbol{\theta}})$, Assumption B entails that

[TABLE]

under ${\rm P}^{(n)}_{\boldsymbol{\theta}}$, ${\boldsymbol{\theta}}\in{\boldsymbol{\Theta}}_{0}$. Using Assumption B again, we have

[TABLE]

under ${\rm P}^{(n)}_{\boldsymbol{\theta}}$, ${\boldsymbol{\theta}}\in{\boldsymbol{\Theta}}_{0}$, with

[TABLE]

Thus, the third Le Cam Lemma (jointly with the fact that (E.20) also holds under ${\rm P}^{(n)}_{{\boldsymbol{\theta}}_{n}}$, from contiguity) directly yields that, under ${\rm P}^{(n)}_{{\boldsymbol{\theta}}_{n}}$,

[TABLE]

where $\tilde{\bf F}$ is obtained from ${\bf F}$ by deleting its last column and last row.

Therefore, conditional on ${\bf D}^{(n)}={\bf D}$,

[TABLE]

under ${\rm P}^{(n)}_{{\boldsymbol{\theta}}_{n}}$, where we let

[TABLE]

and

[TABLE]

with ${\bf L}_{\boldsymbol{\theta}}={\boldsymbol{\Sigma}}_{\boldsymbol{\theta}}{\bf C}_{\boldsymbol{\theta}}^{\prime}({\bf C}_{\boldsymbol{\theta}}{\boldsymbol{\Sigma}}_{\boldsymbol{\theta}}{\bf C}_{\boldsymbol{\theta}}^{\prime})^{-}{\bf C}_{\boldsymbol{\theta}}{\boldsymbol{\Sigma}}_{\boldsymbol{\theta}}$. The result then follows from the fact that ${\boldsymbol{\nu}}_{n}^{-1}(\hat{\boldsymbol{\theta}}_{\rm PTE}-{\boldsymbol{\theta}}_{n})=(1-\lambda(\|{\bf D}\|^{2})){\boldsymbol{\nu}}_{n}^{-1}(\hat{{\boldsymbol{\theta}}}_{\rm U}-{\boldsymbol{\theta}}_{n})+\lambda(\|{\bf D}\|^{2}){\boldsymbol{\nu}}_{n}^{-1}(\hat{{\boldsymbol{\theta}}}_{\rm C}-{\boldsymbol{\theta}}_{n})$ (by using the identities $\lambda^{2}(v)=\lambda(v)$, $(1-\lambda(v))^{2}=1-\lambda(v)$, and $\lambda(v)(1-\lambda(v))=0$). $\square$

The proof of Proposition 1 requires the following preliminary result.

Lemma 1** (Saleh (2006), pp. 32).**

Let ${\bf Z}$ be a Gaussian random $p$-vector with mean vector ${\boldsymbol{\mu}}$ and covariance matrix ${\bf I}_{p}$. Then, for any real measurable function $\varphi$,

[TABLE]

and

[TABLE]

where $V\sim\chi^{2}_{p+2}(\|{\boldsymbol{\mu}}\|^{2})$ and $W\sim\chi^{2}_{p+4}(\|{\boldsymbol{\mu}}\|^{2})$.

Proof of Proposition 1. Since ${\rm E}[{\bf D}]={\bf P}^{\perp}_{{\boldsymbol{\Upsilon}},{\rm eff}}{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{1/2}{\boldsymbol{\tau}}$ and since ${\bf P}_{{\boldsymbol{\Upsilon}},{\rm eff}}$ is idempotent, we have

[TABLE]

Since ${\bf P}_{{\boldsymbol{\Upsilon}},{\rm eff}}^{\perp}$ is a projection matrix with rank $p-r$, it decomposes into ${\bf P}_{{\boldsymbol{\Upsilon}},{\rm eff}}^{\perp}={\bf O}{\boldsymbol{\Lambda}}{\bf O}^{\prime}$, where ${\bf O}$ is a $p\times p$ orthogonal matrix and ${\boldsymbol{\Lambda}}:={\rm diag}(1,\ldots,1,0,\ldots,0)$ is a diagonal matrix with ${\rm tr}[{\boldsymbol{\Lambda}}]=p-r$. The random vector ${\bf E}:={\bf O}^{\prime}{\bf D}$ is then Gaussian with mean vector ${\boldsymbol{\Lambda}}{\bf O}^{\prime}{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{1/2}{\boldsymbol{\tau}}$ and covariance matrix ${\boldsymbol{\Lambda}}$. Lemma 1(i) thus entails that

[TABLE]

where $\gamma_{2}$ is based on a non-central chi-square distribution with $p-r+2$ degrees of freedom and non-centrality parameter $({\boldsymbol{\Lambda}}{\bf O}^{\prime}{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{1/2}{\boldsymbol{\tau}})^{\prime}{\boldsymbol{\Lambda}}{\bf O}^{\prime}{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{1/2}{\boldsymbol{\tau}}={\boldsymbol{\tau}}^{\prime}{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{1/2}{\bf P}_{{\boldsymbol{\Upsilon}},{\rm eff}}^{\perp}{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{1/2}{\boldsymbol{\tau}}$. Plugging this into (E.21) provides the result for ${\rm E}[{\boldsymbol{\mu}}_{\rm PTE,eff}^{\rm Vic}]$.

We thus turn to ${\rm Var}[{\boldsymbol{\mu}}_{\rm PTE,eff}^{\rm Vic}]$. Since $(1-\lambda(v))^{2}=1-\lambda(v)$, we have

[TABLE]

where we used (E.22). Now, by assumption, ${\rm E}[{\bf D}{\bf D}^{\prime}]={\rm Var}[{\bf D}]+{\rm E}[{\bf D}]({\rm E}[{\bf D}])^{\prime}={\bf P}^{\perp}_{{\boldsymbol{\Upsilon}},{\rm eff}}+{\bf P}^{\perp}_{{\boldsymbol{\Upsilon}},{\rm eff}}{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{1/2}{\boldsymbol{\tau}}{\boldsymbol{\tau}}^{\prime}{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{1/2}{\bf P}^{\perp}_{{\boldsymbol{\Upsilon}},{\rm eff}}$, and, applying Lemma 1(ii) along the same lines as above, we have that ${\rm E}[\lambda(\|{\bf D}\|^{2}){\bf D}{\bf D}^{\prime}]={\bf O}{\rm E}[\lambda(\|{\bf E}\|^{2}){\bf E}{\bf E}^{\prime}]{\bf O}^{\prime}=\gamma_{2}{\bf P}^{\perp}_{{\boldsymbol{\Upsilon}},{\rm eff}}+\gamma_{4}{\bf P}^{\perp}_{{\boldsymbol{\Upsilon}},{\rm eff}}{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{1/2}{\boldsymbol{\tau}}{\boldsymbol{\tau}}^{\prime}{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{1/2}{\bf P}^{\perp}_{{\boldsymbol{\Upsilon}},{\rm eff}}$. Plugging these expressions into (4.2) then provides the result. $\square$

Proof of Proposition 2. Contiguity implies that (2.3) also holds under ${\rm P}^{(n)}_{{\boldsymbol{\theta}}_{n}}$, so that

[TABLE]

under ${\rm P}^{(n)}_{{\boldsymbol{\theta}}_{n}}$. Since Le Cam’s third lemma entails that ${\boldsymbol{\Delta}}_{\boldsymbol{\theta}}^{(n)}$ is asymptotically normal with mean vector ${\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}{\boldsymbol{\tau}}$ and covariance matrix ${\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}$ under ${\rm P}^{(n)}_{{\boldsymbol{\theta}}_{n}}$, it follows that ${\boldsymbol{\nu}}_{n}^{-1}(\hat{\boldsymbol{\theta}}_{\rm U}-{\boldsymbol{\theta}}_{n})$ is asymptotically normal with mean vector ${\bf 0}$ and covariance matrix ${\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{-1}$ under ${\rm P}^{(n)}_{{\boldsymbol{\theta}}_{n}}$, which yields ${\rm AMSE}_{{\boldsymbol{\theta}},{\boldsymbol{\tau}}}(\hat{\boldsymbol{\theta}}_{\rm U})={\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{-1}$. Working along the same lines, we have that, under ${\rm P}^{(n)}_{{\boldsymbol{\theta}}_{n}}$,

[TABLE]

It directly follows that ${\boldsymbol{\nu}}_{n}^{-1}(\hat{\boldsymbol{\theta}}_{\rm C}-{\boldsymbol{\theta}}_{n})$ is, still under ${\rm P}^{(n)}_{{\boldsymbol{\theta}}_{n}}$, asymptotically normal with mean vector ${\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{-1/2}{\bf P}_{{\boldsymbol{\Upsilon}},{\rm eff}}{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{1/2}{\boldsymbol{\tau}}-{\boldsymbol{\tau}}=-{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{-1/2}{\bf P}_{{\boldsymbol{\Upsilon}},{\rm eff}}^{\perp}{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{1/2}{\boldsymbol{\tau}}$ and covariance matrix ${\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{-1/2}{\bf P}_{{\boldsymbol{\Upsilon}},{\rm eff}}{\boldsymbol{\Gamma}}_{\boldsymbol{\theta}}^{-1/2}$. The expression for ${\rm AMSE}_{{\boldsymbol{\theta}},{\boldsymbol{\tau}}}(\hat{\boldsymbol{\theta}}_{\rm C})$ given in Proposition 2 directly follows. $\square$