Directing Power Towards Conic Parameter Subspaces

TL;DR

This paper introduces a new high-dimensional test statistic for specific parameter subspaces that improves power and avoids inverse covariance estimation, applicable to sparse or nearly-sparse vectors.

Contribution

It proposes a novel conic subspace-based test statistic that generalizes Wald tests and is computationally feasible with regularized regression methods.

Findings

Effective in high-dimensional settings with small subspaces

Compatible with $\, ext{l}_0$- and $ ext{l}_1$-regularized regression

Demonstrates improved power over traditional quadratic tests

Abstract

For a high-dimensional parameter of interest, tests based on quadratic statistics are known to have low power against subsets of the parameter space (henceforth, parameter subspaces). In addition, they typically involve an inverse covariance matrix which is difficult to estimate in high-dimensional settings. I simultaneously address these two issues by proposing a novel test statistic that is large in a conic parameter subspace of interest. This test statistic generalizes the Wald statistic and nests many well-known test statistics. For a given parameter subspace, the statistic is free of tuning parameters and suitable for high-dimensional settings if the subspace is sufficiently small. It can be computed using regularized linear regression, where the type of regularization and the regularization parameters are completely determined by the parameter subspace of interest. I illustrate the statistic on subspaces that consist of sparse or nearly-sparse vectors, for which the computation corresponds to $\ell_0$- and $\ell_1$-regularized regression, respectively.

Figures

Tables

Table 1. Table 1: Several test statistics that are nested by the T 𝒞 subscript 𝑇 𝒞 T_{\mathcal{C}} statistic.

$𝒞$	$T_{𝒞}$ statistic
${ℝ}^p$	$\sqrt{{𝒎}^{\prime }{𝑺}^{-1}𝒎}$ (Wald statistic)
${𝒞}_{+}=\left\{𝝀\right|𝝀\ge \mathrm{𝟎}\}$	$T_{+}$ (Koning and Bekker, 2019)
${𝒞}_{\left(1\right)}=\left\{𝝀\right|{\lambda }_j=0,j\ne 1\}$	$\left|m_1\right|/s_1$ (two-sided $t$-statistic)
${𝒞}_{\left(1\right)}\cap {𝒞}_{+}$	${\left(m_1/s_1\right)}^{+}$ (one-sided $t$-statistic)
${𝒞}_1=\left\{𝝀\right|{\Vert 𝝀\Vert }_0\le 1\}$	${\max }_j\left|m_j\right|/s_j$ (Bugni et al., 2016)
${𝒞}_1\cap {𝒞}_{+}$	${\max }_j{(m_j/s_j)}^{+}$ (Chernozhukov et al., 2018)
$\{(-1,1,\mathrm{𝟎}),(1,-1,\mathrm{𝟎})\}$	$\left|m_1-m_2\right|/\sqrt{s_1^2+s_2^2}$, if $𝑺$ is diagonal (Welch, 1947)

Table 2. Table 2: Monte Carlo rejection rates if H 0 : 𝝁 = 𝟎 : subscript 𝐻 0 𝝁 0 H_{0}:\bm{\mu}=\bm{0} is true.

			$T_{𝒞}$			PE	Wald
n	p	$\rho$	$T_1$/$T_1^d$	$T_{20}$	$T_{20}^d$
		0	.051	.038	.055	.786	.056
	100	.5	.054	.047	.061	.450	.045
		.7	.045	.048	.054	.270	.059
		0	.041	.043	.053	.895	.043
30	300	.5	.056	.055	.046	.434	.049
		.7	.045	.041	.063	.274	.047
		0	.042	.052	.050	.950	.055
	500	.5	.040	.045	.044	.459	.062
		.7	.056	.043	.048	.261	.049
		0	.051	.054	.042	.080	.058
	100	.5	.051	.043	.062	.058	.044
		.7	.036	.046	.057	.074	.056
		0	.036	.045	.040	.080	.039
250	300	.5	.039	.053	.055	.077	.052
		.7	.042	.045	.044	.053	.053
		0	.047	.046	.042	.054	.055
	500	.5	.049	.041	.054	.060	.052
		.7	.047	.049	.058	.060	.049

Table 3. Table 3: Monte Carlo rejection rates if H 1 1 : 𝝁 ∈ 𝒟 1 : superscript subscript 𝐻 1 1 𝝁 subscript 𝒟 1 H_{1}^{1}:\bm{\mu}\in\mathcal{D}_{1} is true.

			$T_{𝒞}$			PE	Wald
n	p	$\rho$	$T_1$/$T_1^d$	$T_{20}$	$T_{20}^d$
		0	.682	.068	.477	.985	.053
	100	.5	.775	.128	.085	.972	.059
		.7	.842	.176	.081	.959	.049
		0	.551	.055	.341	.980	.053
30	300	.5	.654	.107	.083	.955	.057
		.7	.792	.147	.073	.938	.045
		0	.498	.064	.305	.991	.056
	500	.5	.630	.091	.085	.949	.052
		.7	.724	.147	.080	.917	.051
		0	.743	.324	.482	.665	.195
	100	.5	.810	.644	.101	.752	.455
		.7	.864	.921	.096	.876	.755
		0	.637	.257	.352	.506	.054
250	300	.5	.741	.453	.099	.473	.056
		.7	.826	.726	.081	.515	.053
		0	.601	.205	.312	.429	.061
	500	.5	.694	.384	.074	.434	.057
		.7	.789	.610	.096	.398	.046

Table 4. Table 4: Monte Carlo rejection rates if H 1 20 : 𝝁 ∈ 𝒟 20 : superscript subscript 𝐻 1 20 𝝁 subscript 𝒟 20 H_{1}^{20}:\bm{\mu}\in\mathcal{D}_{20} is true.

			$T_{𝒞}$			PE	Wald
n	p	$\rho$	$T_1$/$T_1^d$	$T_{20}$	$T_{20}^d$
		0	.374	.102	.883	.982	.045
	100	.5	.258	.118	.217	.796	.053
		.7	.280	.162	.201	.567	.048
		0	.205	.060	.549	.979	.051
30	300	.5	.176	.072	.119	.701	.045
		.7	.196	.106	.136	.510	.053
		0	.159	.064	.409	.980	.060
	500	.5	.139	.085	.102	.698	.053
		.7	.182	.088	.101	.466	.050
		0	.291	.337	.752	.387	.315
	100	.5	.203	.537	.130	.622	.607
		.7	.237	.779	.155	.880	.898
		0	.161	.152	.387	.094	.052
250	300	.5	.141	.288	.093	.080	.051
		.7	.175	.473	.105	.089	.038
		0	.145	.107	.287	.088	.041
	500	.5	.123	.206	.071	.085	.036
		.7	.138	.325	.102	.051	.057

Equations

$${\mathbf{y}}_n=\mathbf{\mu }+\mathbf{B}{\mathbf{f}}_n+{\mathbf{\epsilon }}_n.$$

$$\Vert \mathbf{x}{\Vert }_q=\{\begin{array}{ll}{\sum }_j{1}_{\{x_j\ne 0\}},& q=0,\\ ({\sum }_j|x_j{|}^q{)}^{\frac{1}{q}},& 0<q<\infty .\end{array}$$

$$T=\sqrt{{\mathbf{m}}^{\prime }{\mathbf{S}}^{-1}\mathbf{m}},$$

$$T=\underset{{\mathbf{\theta }}^{\prime }\mathbf{\theta }=1}{\max }{\mathbf{m}}^{\prime }{\mathbf{S}}^{-\frac{1}{2}}\mathbf{\theta }=\underset{\mathbf{\lambda }\mathbf{S}\mathbf{\lambda }=1}{\max }{\mathbf{m}}^{\prime }{\mathbf{S}}^{-\frac{1}{2}}{\mathbf{S}}^{\frac{1}{2}}\mathbf{\lambda }=\underset{\mathbf{\lambda }\in \mathcal{S}}{\max }{\mathbf{m}}^{\prime }\mathbf{\lambda }=\underset{\mathbf{\lambda }\in {\mathcal{S}}^0}{\max }{\mathbf{m}}^{\prime }\mathbf{\lambda },$$

$$\sqrt{{\mathbf{x}}^{\prime }\mathbf{x}}=\underset{\begin{array}{c}\mathbf{\theta }\end{array}}{\max }\frac{{\mathbf{x}}^{\prime }\mathbf{\theta }}{\sqrt{{\mathbf{\theta }}^{\prime }\mathbf{\theta }}}=\underset{\begin{array}{c}\mathbf{\theta }\\ {\mathbf{\theta }}^{\prime }\mathbf{\theta }=1\end{array}}{\max }{\mathbf{x}}^{\prime }\mathbf{\theta }.$$

$$T_{\mathcal{C}}=\underset{\begin{array}{c}\mathbf{\lambda }\in {\mathcal{S}}^0\cap \mathcal{C}\end{array}}{\max }{\mathbf{m}}^{\prime }\mathbf{\lambda },$$

$$T_{{\mathcal{C}}^{\varnothing }}=\underset{\begin{array}{c}\mathbf{\lambda }\in {\mathcal{C}}^{\varnothing }\end{array}}{\max }\frac{{\mathbf{m}}^{\prime }\mathbf{\lambda }}{\sqrt{{\mathbf{\lambda }}^{\prime }\mathbf{S}\mathbf{\lambda }}}=\underset{\begin{array}{c}\mathbf{\lambda }\in \mathcal{C}\\ {\mathbf{m}}^{\prime }\mathbf{\lambda }=1\end{array}}{\max }\frac{1}{\sqrt{{\mathbf{\lambda }}^{\prime }\mathbf{S}\mathbf{\lambda }}}=\frac{1}{{\min }_{\begin{array}{c}\mathbf{\lambda }\in \mathcal{C}\\ {\mathbf{m}}^{\prime }\mathbf{\lambda }=1\end{array}}\sqrt{{\mathbf{\lambda }}^{\prime }\mathbf{S}\mathbf{\lambda }}}.$$

$$\hat{\mathbf{\lambda }}=\frac{\hat{\mathbf{\beta }}}{\sqrt{{\hat{\mathbf{\beta }}}^{\prime }\mathbf{S}\hat{\mathbf{\beta }}}},$$

$${\mathbf{m}}^{\prime }\hat{\mathbf{\lambda }}={\Vert \sqrt{\frac{{\hat{\mathbf{\lambda }}}^{\prime }\hat{\mathbf{\lambda }}}{{\hat{\mathbf{\lambda }}}^{\prime }\mathbf{S}\hat{\mathbf{\lambda }}}}{\mathbf{P}}_{\hat{\mathbf{\lambda }}}\mathbf{m}\Vert }_2,$$

$$H_0:\mathbf{\mu }\in {\mathcal{C}}^{\circ }.$$

$$T_{\mathcal{C}}^{\ast }=\underset{\begin{array}{c}\mathbf{\lambda }\in \mathcal{C}\\ {\mathbf{\lambda }}^{\prime }\Sigma \mathbf{\lambda }=1\end{array}}{\max }{\mathbf{\mu }}^{\prime }\mathbf{\lambda }=\underset{\begin{array}{c}{\Sigma }^{-\frac{1}{2}}\mathbf{\lambda }\in \mathcal{C}\\ {\mathbf{\lambda }}^{\prime }\mathbf{\lambda }=1\end{array}}{\max }{\mathbf{\mu }}^{\prime }{\Sigma }^{-\frac{1}{2}}\mathbf{\lambda }=\underset{\begin{array}{c}\mathbf{\lambda }\in {\Sigma }^{\frac{1}{2}}\mathcal{C}\\ {\mathbf{\lambda }}^{\prime }\mathbf{\lambda }=1\end{array}}{\max }{\mathbf{\mu }}^{\prime }{\Sigma }^{-\frac{1}{2}}\mathbf{\lambda },$$

$$H_1^{\ast }:\mathbf{\mu }\in \Sigma {\mathcal{C}}^{\varnothing }.$$

$$H_1:\mathbf{\mu }\in {\mathcal{C}}^{\varnothing }.$$

$$\hat{\mathbf{\lambda }}={\mathbf{S}}^{-\frac{1}{2}}\hat{\mathbf{\beta }}/\sqrt{{\hat{\mathbf{\beta }}}^{\prime }\hat{\mathbf{\beta }}},$$

$$\mathbf{X}={\mathbf{\iota }}_n{\mathbf{\mu }}^{\prime }+\mathbf{E},$$

$$H_0$$

$$H_1^k$$

$$\hat{\mathbf{\lambda }}=\hat{\mathbf{\beta }}/\sqrt{{\hat{\mathbf{\beta }}}^{\prime }\hat{\Sigma }\hat{\mathbf{\beta }}},$$

$$\hat{\mathbf{\beta }}=\underset{\begin{array}{c}\mathbf{\beta }\\ \Vert \mathbf{\beta }{\Vert }_0\le k\end{array}}{\mathrm{argmin}}\Vert {\mathbf{\iota }}_n-\mathbf{X}\mathbf{\beta }{\Vert }_2^2.$$

$$\underset{\begin{array}{c}\mathbf{\beta }\\ \Vert \mathbf{\beta }{\Vert }_0\le k\end{array}}{\min }\Vert {\mathbf{S}}^{-\frac{1}{2}}\hat{\mathbf{\mu }}-\mathbf{\beta }{\Vert }_2^2.$$

$$T_k^d=\sqrt{{\hat{\mathbf{\mu }}}^{\prime }{\mathbf{S}}^{-1}{\hat{\mathbf{\mu }}}_{{\mathcal{J}}_k}}.$$

$$J_0=\sqrt{p}{\hat{\mathbf{\mu }}}^{\prime }\text{diag}(\hat{\Sigma }{)}^{-1}{\hat{\mathbf{\mu }}}_{\mathcal{J}},$$

$$H_1:\mathbf{\mu }\in {\mathcal{C}}_{{\Lambda }_t}^{\varnothing }.$$

$$\hat{\mathbf{\beta }}=\underset{\begin{array}{c}\mathbf{\beta }\\ \Vert \mathbf{\beta }{\Vert }_1\le t/\sqrt{{\hat{\mathbf{\beta }}}^{\prime }\hat{\Sigma }\hat{\mathbf{\beta }}}\end{array}}{\mathrm{argmin}}\Vert {\mathbf{\iota }}_n-\mathbf{X}\mathbf{\beta }{\Vert }_2^2,$$

$$b_{n,s}=\{\begin{array}{l}0.75,\text{ if }n=30,s=1,\\ 0.25,\text{ if }n=30,s=20,\\ 0.25,\text{ if }n=250,s=1,\\ 0.07,\text{ if }n=250,s=20.\end{array}$$

$$J_0=\sqrt{p}{\mathbf{m}}^{\prime }\mathrm{diag}(\hat{\Sigma }{)}^{-1}{\mathbf{m}}_{\mathcal{J}},$$

$$\underset{\begin{array}{c}\mathbf{\lambda }\in \mathcal{C}\\ {\mathbf{\lambda }}^{\prime }\mathbf{S}\mathbf{\lambda }={\gamma }^2\end{array}}{\mathrm{argmax}}{\mathbf{m}}^{\prime }\mathbf{\lambda }=\gamma \underset{\begin{array}{c}\mathbf{\lambda }\in \mathcal{C}\\ {\mathbf{\lambda }}^{\prime }\mathbf{S}\mathbf{\lambda }=1\end{array}}{\mathrm{argmax}}{\mathbf{m}}^{\prime }\mathbf{\lambda }.$$

$$\underset{\begin{array}{c}\mathbf{\lambda }\in \mathcal{C}\\ {\mathbf{\lambda }}^{\prime }\mathbf{S}\mathbf{\lambda }={\gamma }^2\end{array}}{\mathrm{argmax}}{\mathbf{m}}^{\prime }\mathbf{\lambda }$$

$$\hat{\mathbf{\lambda }}=\frac{\hat{\mathbf{\beta }}}{\sqrt{{\hat{\mathbf{\beta }}}^{\prime }\mathbf{S}\hat{\mathbf{\beta }}}},$$

$$\hat{\mathbf{\lambda }}$$

$$=\frac{1}{\sqrt{{\hat{\mathbf{\beta }}}^{\prime }\mathbf{S}\hat{\mathbf{\beta }}}}\underset{\begin{array}{c}\mathbf{\lambda }\in \mathcal{C}\\ {\mathbf{\lambda }}^{\prime }\mathbf{S}\mathbf{\lambda }={\hat{\mathbf{\beta }}}^{\prime }\mathbf{S}\hat{\mathbf{\beta }}\end{array}}{\mathrm{argmin}}1-2{\mathbf{m}}^{\prime }\mathbf{\lambda }+{\mathbf{\lambda }}^{\prime }\mathbf{S}\mathbf{\lambda }=\frac{1}{\sqrt{{\hat{\mathbf{\beta }}}^{\prime }\mathbf{S}\hat{\mathbf{\beta }}}}\hat{\mathbf{\beta }},$$

Full text

PRELIMINARY VERSION

12 November 2019

1 Introduction

This paper concerns testing hypotheses on the location of a high-dimensional parameter $\bm{\mu}$, whose dimension $p$ may exceed the number of observations $n$. Traditionally, such tests are often based on a quadratic statistic like the Wald statistic. However, tests based on quadratic statistics are known to have low power against subsets of the parameter space (henceforth, parameter subspaces) in high dimensions. This has provoked a recent interest in constructing tests that direct power towards parameter subspaces of interest (see e.g. Fan et al., 2015; Kock and Preinerstorfer, 2019).

Such parameter subspaces arise if the econometrician has some prior information about the nature of violations of the null hypothesis. For example, suppose that we want to test the null hypothesis $H_{0}:\bm{\mu}=\bm{0}$, describing a collection of $p$ equalities. Then one may have prior information that if $H_{0}$ is false, only some number $k<p$ of these equalities are violated. That is, the parameter $\bm{\mu}$ is expected to be $k$-sparse under the alternative (i.e. containing $p-k$ zeros). So, we may want to use a test that directs power towards the parameter subspace consisting of $k$-sparse vectors. The following example describes how such situations can arise in practice.

Example 1**.**

Multi-factor pricing models describe the excess returns of assets as a linear combination of a limited number of factors. In particular, for a collection of $p$ assets at time $n$, let $\bm{y}_{n}$ be the $p$-vector of excess returns, let $\bm{f}_{n}$ be the $m$-vector containing the $m$ observable factors and let $\bm{\varepsilon}_{n}$ be a $p$-vector of errors. Let $\bm{B}$ denote the $p\times m$ matrix of factor loadings, and $\bm{\mu}$ the $p$-vector of asset-specific returns that are not captured by the factors (also known as ‘alpha’ in the finance literature). Then, a multi-factor pricing model can be written as

[TABLE]

If the factor model is well specified, efficient pricing theory suggests that the excess returns should be fully captured by the factors so that $\bm{\mu}=\bm{0}$. To assess this theory, one may want to test the null hypothesis $H_{0}:\bm{\mu}=\bm{0}$. As such models are well founded in theory, one may expect that if this hypothesis is false then it would only be violated by a small number of exceptional assets. So, if the alternative hypothesis holds, then the underlying vector of excess returns is likely to be sparse. Therefore, one may want to use a test that directs power towards a parameter subspace that consists of sparse vectors.

Besides low power against parameter subspaces, another problem with quadratic statistics is that they typically involve and inverse covariance matrix, which is usually not known in practice and must be estimated. This is problematic in high-dimensional settings, as standard estimators of such a covariance matrix are not invertible if $p>n$. Without imposing additional assumptions, this means that the Wald statistic does not exist. The solution provided in the literature is to estimate the covariance matrix by imposing restrictions on its structure, through regularization or other means (see e.g. Srivastava and Du, 2008; Chen et al., 2011). However, if no information about the covariance matrix is available, then it is not clear which restrictions should be used. Imposing the wrong restrictions may lead to poor estimates, which in turn may lead to a loss of power.

I simultaneously address these two issues by proposing a novel test statistic that is large in a conic parameter subspace that is specified by the econometrician.111A cone $\mathcal{C}$ is a collection of vectors that is closed under positive scalar multiplication. That is, if $\bm{\lambda}\in\mathcal{C}$, then $\gamma\bm{\lambda}\in\mathcal{C}$, for all $\gamma>0$. This test statistic is a generalization of the Wald statistic. It nests many well-known test statistics that can be recovered by choosing a particular parameter subspace of interest.

For the test statistic to exist, a restricted eigenvalue condition must be satisfied. The strength of this condition depends directly on the parameter subspace of interest. The condition is weaker than the condition that the sample covariance matrix is positive-definite. Unlike the standard Wald statistic, the test statistic can therefore be used even in high-dimensional settings, as long as the parameter subspace of interest is sufficiently ‘small’.

In addition, I show that the computation of the test statistic can be formulated as a quadratic minimization problem, where the arguments over which is minimized are restricted to the parameter subspace of interest. If the standard sample covariance matrix and sample mean are used as estimators, this problem reduces to regularized linear regression with a constant dependent variable. I provide an additional result for the special case of diagonal estimators of the covariance matrix and parameter subspaces defined by sign or sparsity restrictions. In that case, the computation reduces to solving a minimum distance problem which can lead to a closed-form test statistic.

The test statistic is illustrated with the parameter subspace that consists of all $k$-sparse vectors. In case of a $k$-sparse parameter subspace, the existence criterion reduces to the requirement that the covariance estimator is at least of rank $k$. This condition is typically satisfied if $k<n$. The computation of the test statistic is a special case of best-subset selection, also known as $\ell_{0}$-regularization. Best-subset selection has seen a recent surge of interest after Bertsimas et al. (2016) showed that this problem is easier to solve than was commonly assumed, by exploiting gradient descent methods and commercially available mixed-integer optimization solvers.222See e.g. Hastie et al. (2017); Mazumder et al. (2017); Hazimeh and Mazumder (2018); Koning and Bekker (2018) for more recent work. If one uses a diagonal covariance matrix estimator, then the test statistic has a closed form and is related to threshold-type statistics (see e.g., Fan, 1996; Zhong et al., 2013).

In addition, I recover the parameter subspace for which the computation of the test statistic coincides with Lasso, which is another popular technique used to obtain sparse solutions (Tibshirani, 1996). This parameter subspace consists of a union of convex cones that are centered around sparse vectors, whose widths depend on the Lasso regularization parameter. Therefore, the parameter subspace corresponding to Lasso can be interpreted as the collection of vectors that are ‘close’ to a sparse vector. Intuitively, these may be used if one suspects only a limited number of large violations of the null hypothesis $H_{0}:\bm{\mu}=\bm{0}$ might occur.

For the illustration of the test statistic with sparse and nearly-sparse cones, I construct a critical value using reflection-based randomization (see e.g. Lehmann and Romano, 2006). While this construction relies on a symmetry assumption on the distribution of the error term, the advantage is that size control is guaranteed in small samples, even if $p\gg n$.

The power properties of the tests proposed in this paper are studied using Monte Carlo simulation. The results suggest that the tests perform favorably compared to the Wald test if $p<n$ and retains power even if $p\gg n$. In addition, the test typically outperforms the power enhancement technique of Fan et al. (2015).

The power enhancement technique is a recently proposed method that increases power in a parameter subspaces of interest. This technique works as follows: given some initial test, one chooses an enhancement test which has asymptotic size zero and is consistent against a parameter subspace, against which the initial test is not consistent. The combination of these two tests has asymptotic size equal to the initial test, and may be consistent against a strictly larger set of alternatives. The existence of power enhancement tests has recently been discussed by Kock and Preinerstorfer (2019).

The main advantage of the methodology proposed in this paper over the power enhancement technique is that it is constructive: when the econometrician specifies a parameter subspace of interest, this simultaneously defines a test statistic. For the power enhancement technique, both an initial and enhancement test must be chosen by the econometrician. A typical choice for the initial test, as used by Fan et al. (2015), would be based on a Wald-type statistic, which suffers from the issues in high-dimensional settings that were described above. Finding an appropriate enhancement test for a given parameter subspace may also not be straightforward in practice.

The advantage of the power enhancement technique is its attractive asymptotic properties, which promise a strictly larger subset of the parameter space against which the test is consistent while maintaining asymptotic size. It remains unclear how this translates to finite samples. Fan et al. (2015) conduct several numerical experiments that suggest small size distortions may occur in finite samples. Larger size distortions are found in the Monte Carlo simulations presented in Section 5. In order to compensate for the size distortion, one could reduce the size of the initial test. This would effectively result in a test that directs power towards the subspace of interest by taking power away from the complement of this subspace.

1.1 Notation

For the remainder of the paper, the following notation is used. For a set $\mathcal{A}$, the set $\mathcal{A}^{0}=\mathcal{A}\cup\{\bm{0}\}$ denotes the union of the set and the origin. The set $\mathcal{A}^{\emptyset}=\mathcal{A}\setminus\{\bm{0}\}$ excludes the origin. Let $\bm{I}_{n}=(\bm{e}_{1},\bm{e}_{2},\dots,\bm{e}_{n})$ denote the $n\times n$ identity matrix, and $\bm{\iota}_{n}$ an $n$-vector of ones. For a matrix $\bm{X}$ with full column rank, the matrix $\bm{P}_{\bm{x}}=\bm{X}(\bm{X}^{\prime}\bm{X})^{-1}\bm{X}^{\prime}$ denotes the projection matrix of $\bm{X}$. For a vector $\bm{x}$ with elements $x_{j}$, $j=1,\dots,p$, the subscripted vector $\bm{x}_{\mathcal{J}}$ has $j$th element equal to $x_{j}$ if $j\in\mathcal{J}$ and 0, otherwise. Finally, the following standard notation is used

[TABLE]

2 From Quadratic to Conic statistics

Let $\bm{\mu}$ be the $p$-vector of interest from the parameter space $\mathbb{R}^{p}$, and let $\bm{m}$ be an estimator for $\bm{\mu}$ with covariance $\bm{\varSigma}>0$. The matrix $\bm{S}\geq 0$ denotes a generic estimator of $\bm{\varSigma}$. If $\bm{S}$ is positive definite, a quadratic or Wald-type statistic that is typically used for testing the hypothesis $H_{0}:\bm{\mu}=\bm{0}$ is of the form

[TABLE]

which is a weighted Euclidean norm of $\bm{m}$. Let us slightly extend its definition so that $T=0$ if $\bm{m}=\bm{0}$ and $\bm{S}$ is singular. Then, $T$ can be equivalently written as a maximization problem. Specifically, let $\mathcal{S}=\{\bm{\lambda}\in\mathbb{R}^{p}\ |\ \bm{\lambda}^{\prime}\bm{S}\bm{\lambda}=1\}$ denote the unit ellipsoid induced by $\bm{S}$, then

[TABLE]

where the first step follows from the Cauchy-Schwarz inequality.333By the Cauchy-Schwarz inequality with $\bm{\theta}\neq\bm{0}$, we have $\bm{x}^{\prime}\bm{\theta}\leq\sqrt{\bm{x}^{\prime}\bm{x}}\sqrt{\bm{\theta}^{\prime}\bm{\theta}}$, so that $\bm{x}^{\prime}\bm{\theta}/\sqrt{\bm{\theta}^{\prime}\bm{\theta}}\leq\sqrt{\bm{x}^{\prime}\bm{x}}$, which implies

$\sqrt{\bm{x}^{\prime}\bm{x}}=\max_{\begin{subarray}{c}\bm{\theta}\end{subarray}}\frac{\bm{x}^{\prime}\bm{\theta}}{\sqrt{\bm{\theta}^{\prime}\bm{\theta}}}=\max_{\begin{subarray}{c}\bm{\theta}\\ \bm{\theta}^{\prime}\bm{\theta}=1\end{subarray}}\bm{x}^{\prime}\bm{\theta}.$

Notice that the right-hand side reformulation does not use the inverse of $\bm{S}$, but the equality holds as the maximization is unbounded if $\bm{S}$ is singular and $\bm{m}\neq\bm{0}$.

The key idea in this paper is to extend the definition of $T$ by restricting the weights $\bm{\lambda}$ to a section of $\mathcal{S}$ that represents a direction of interest. The resulting statistic may exist even if $\bm{S}$ is singular. Specifically, let $\mathcal{C}\subseteq\mathbb{R}^{p}$ be a (not necessarily convex) cone and assume that $\mathcal{C}$ is closed in $\mathbb{R}^{p}$. Then, I propose the following ‘conic’ statistic

[TABLE]

which is an asymmetric semi-norm of $\bm{m}$. The dependence on $\bm{m}$ and $\mathcal{S}$ is suppressed for notational convenience. Table 1 contains an overview of several test statistics that are nested by $T_{\mathcal{C}}$ and can be obtained for different choices of $\mathcal{C}$. Notice that $T=T_{\mathbb{R}^{p}}$.

2.1 Existence of $T_{\mathcal{C}}$

While the quadratic statistic $T_{\mathbb{R}^{p}}$ requires $\bm{S}$ to be positive definite in order to exist, $T_{\mathcal{C}}$ typically requires a weaker condition. Specifically, the following restricted eigenvalue condition (see e.g. Van De Geer et al., 2009) guarantees that $T_{\mathcal{C}}$ exists.444The condition follows from first observing that the case $\bm{\lambda}=\bm{0}$ is irrelevant, and then rewriting

$\displaystyle T_{\mathcal{C}^{\emptyset}}=\max_{\begin{subarray}{c}\bm{\lambda}\in\mathcal{C}^{\emptyset}\end{subarray}}\frac{\bm{m}^{\prime}\bm{\lambda}}{\sqrt{\bm{\lambda}^{\prime}\bm{S}\bm{\lambda}}}=\max_{\begin{subarray}{c}\bm{\lambda}\in\mathcal{C}\\ \bm{m}^{\prime}\bm{\lambda}=1\end{subarray}}\frac{1}{\sqrt{\bm{\lambda}^{\prime}\bm{S}\bm{\lambda}}}=\frac{1}{\min_{\begin{subarray}{c}\bm{\lambda}\in\mathcal{C}\\ \bm{m}^{\prime}\bm{\lambda}=1\end{subarray}}\sqrt{\bm{\lambda}^{\prime}\bm{S}\bm{\lambda}}}.$

Condition 1**.**

$\min_{\begin{subarray}{c}\bm{\lambda}\in\mathcal{C}\\ \bm{m}^{\prime}\bm{\lambda}=1\end{subarray}}\bm{\lambda}^{\prime}\bm{S}\bm{\lambda}>0$.

Notice this condition reduces to $\bm{\lambda}^{\prime}\bm{S}\bm{\lambda}>0$ for all $\bm{\lambda}\in\mathbb{R}^{p}$, if $\mathcal{C}=\mathbb{R}^{p}$ (and $\bm{m}\neq\bm{0}$). This is equivalent to $\bm{S}>0$, which indeed coincides with the condition for the typical Wald-type statistic $T_{\mathbb{R}^{p}}$ to exist.

2.2 Computation

The following result shows that $T_{\mathcal{C}}$ can be computed using restricted quadratic minimization. This is convenient as some of such problems have been well studied in the optimization literature. A proof of the result can be found in Appendix A.

Proposition 1**.**

Let $\bm{\lambda}^{\prime}\bm{S}\bm{\lambda}>0$ for all $\bm{\lambda}\in\mathcal{C}^{\emptyset}$. Let $\widehat{\bm{\beta}}=\operatorname*{arg\,min}_{\bm{\beta}\in\mathcal{C}}1-2\bm{m}^{\prime}\bm{\beta}+\bm{\beta}^{\prime}\bm{S}\bm{\beta}$ and $\widehat{\bm{\lambda}}=\operatorname*{arg\,max}_{\begin{subarray}{c}\bm{\lambda}\in\mathcal{S}^{0}\cap\mathcal{C}\end{subarray}}\bm{m}^{\prime}\bm{\lambda}$ be unique optimizers. Then

[TABLE]

if $\widehat{\bm{\beta}}\neq\bm{0}$ and $\widehat{\bm{\lambda}}=\bm{0}$, otherwise.

In Section 4, I discuss a case where the computation reduces to $\ell_{0}$- and $\ell_{1}$-regularized regression. Section 3.3 covers the special case if $\bm{S}$ is diagonal, where the computation simplifies for cones that are described by sign and sparsity restriction.

2.3 Geometric interpretation

Condition 1 can be shown visually using the following geometric interpretation of $T_{\mathcal{C}}$. Suppose that the vector $\widehat{\bm{\lambda}}$ with $\widehat{\bm{\lambda}}^{\prime}\bm{S}\widehat{\bm{\lambda}}\neq 0$ is the maximizing vector. Then $T_{\mathcal{C}}=\bm{m}^{\prime}\widehat{\bm{\lambda}}$ can be viewed as the Euclidean norm of the vector $\bm{m}$, after projection onto $\widehat{\bm{\lambda}}$ and scaling by the relative stretching of $\mathcal{S}$ compared to the sphere:

[TABLE]

where $\bm{P}_{\widehat{\bm{\lambda}}}=\widehat{\bm{\lambda}}\widehat{\bm{\lambda}}^{\prime}/\widehat{\bm{\lambda}}^{\prime}\widehat{\bm{\lambda}}$ is the projection matrix of $\widehat{\bm{\lambda}}$. A visual demonstration for both a singular and positive definite matrix $\bm{S}$ is given in Figure 1.

In the left panel of Figure 1, we see that for the singular $\bm{S}$, the ellipsoid $\mathcal{S}$ is infinitely stretched along the scalar multiples $\gamma\bm{\lambda}_{0}$ of the vector $\bm{\lambda}_{0}=(-1,1)^{\prime}$, for which $\gamma^{2}\bm{\lambda}_{0}^{\prime}\bm{S}\bm{\lambda}_{0}=0$. The chosen cone $\mathcal{C}=\{\bm{\lambda}\ |\ \bm{\lambda}\geq\bm{0}\}$ does not contain the positive or negative scalar multiples of $\bm{\lambda}_{0}$. So, $\bm{\lambda}^{\prime}\bm{S}\bm{\lambda}>0$ for all $\bm{\lambda}\in\mathcal{C}^{\emptyset}$. Therefore, Condition 1 is satisfied which means $T_{\mathcal{C}}$ exists. The value of $T_{\mathcal{C}}$ is represented as the length of the vector with the red diamond at its tip.

In the right panel of Figure 1, $\bm{S}$ is positive definite so that $\mathcal{S}$ is a proper ellipse and Condition 1 is satisfied regardless of $\mathcal{C}$. For the setting that is portrayed, it happens that the unconstrained maximizing vector $\widehat{\bm{\lambda}}$ are already contained in $\mathcal{C}$. Therefore, $T_{\mathcal{C}}=T_{\mathbb{R}^{p}}$. If $\widehat{\bm{\lambda}}$ was not contained in $\mathcal{C}$, then $T_{\mathcal{C}}\leq T_{\mathbb{R}^{p}}$.

3 Hypothesis testing

In this section, $T_{\mathcal{C}}$ is used as a test statistic. In particular, I describe the hypotheses for which $T_{\mathcal{C}}$ would be a suitable test statistic.

3.1 Null hypothesis

I start with describing the null hypothesis, for which the following notation is used. Let $T_{\mathcal{C}}^{*}=\max_{\begin{subarray}{c}\bm{\lambda}\in\mathcal{V}\cap\mathcal{C}\end{subarray}}\bm{\mu}^{\prime}\bm{\lambda}$ denote the population equivalent of $T_{\mathcal{C}}$, where $\mathcal{V}^{0}=\{\bm{y}\ |\ \bm{y}^{\prime}\bm{\varSigma}\bm{y}=1\}$. Let $\mathcal{C}^{\circ}$ denote the polar cone of $\mathcal{C}$, defined by $\mathcal{C}^{\circ}=\{\bm{\mu}\ |\ \bm{\mu}^{\prime}\bm{\lambda}\leq 0,\forall\bm{\lambda}\in\mathcal{C}\}$. The polar cone of $\mathcal{C}$ consists of the vectors that have a right angle or larger angle with any vector in $\mathcal{C}$. It can therefore be interpreted as the cone that ‘points away’ from $\mathcal{C}$.

Notice that the polar cone can equivalently be defined as $\mathcal{C}^{\circ}=\{\bm{\mu}\ |\ \max_{\bm{\lambda}\in\mathcal{V}^{0}\cap\mathcal{C}}\bm{\mu}^{\prime}\bm{\lambda}=0\}$. So, the polar cone $\mathcal{C}^{\circ}$ corresponds exactly to the parameter subspace where $T_{\mathcal{C}}^{*}=0$. Therefore, it seems appropriate to use $T_{\mathcal{C}}$ as a test statistic for the null hypothesis

[TABLE]

Two examples are provided of different null hypotheses that can be constructed using different cones $\mathcal{C}$.

Example 2**.**

The cone $\mathcal{C}_{+}=\{\bm{\lambda}\ |\ \bm{\lambda}\geq\bm{0}\}$ has polar cone $\mathcal{C}_{+}^{\circ}=\{\bm{\mu}\ |\ \bm{\mu}\leq\bm{0}\}$, producing the multi-dimensional ‘one-sided’ null hypothesis $H_{0}:\bm{\mu}\leq\bm{0}$.

Example 3**.**

The cone $\mathcal{C}_{(1)}=\{\bm{\lambda}\ |\ \lambda_{j}=0,\ j\neq 1\}$ has polar cone $\mathcal{C}_{(1)}^{\circ}=\{\bm{\mu}\ |\ \mu_{1}=0\}$ corresponding to the one-dimensional simple hypothesis $H_{0}:\mu_{1}=0$. The intersection $\mathcal{C}_{(1)+}=\mathcal{C}_{(1)}\cap\mathcal{C}_{+}$ has polar cone $\mathcal{C}_{(1)+}^{\circ}$, which constructs the one-dimensional one-sided hypothesis $H_{0}:\mu_{1}\leq 0$.

3.2 Choosing a specific alternative hypothesis

A key observation is that different cones $\mathcal{C}$ can share the same polar cone, and therefore produce the same null hypothesis. This is illustrated in the following example.

Example 4**.**

Let us consider $\mathcal{C}_{1}=\{\bm{\lambda}\ |\ \|\bm{\lambda}\|_{0}\leq 1\}$, containing the cones that coincide with the axes, and the cone $\mathbb{R}^{p}$ that contains the entire parameter space. Notice that $T_{\mathcal{C}_{1}}^{*}=\max_{\bm{\lambda}\in\mathcal{V}^{0}\cap\mathcal{C}_{1}}\bm{\mu}^{\prime}\bm{\lambda}=\max_{j}|\mu_{j}|/\sigma_{j}$ and $T_{\mathbb{R}^{p}}^{*}=\max_{\bm{\lambda}\in\mathcal{V}^{0}}\bm{\mu}^{\prime}\bm{\lambda}=\sqrt{\bm{\mu}^{\prime}\bm{\varSigma}^{-1}\bm{\mu}}$, are zero if and only if $\bm{\mu}=\bm{0}$. Therefore, the polar cone corresponding to both $\mathcal{C}_{1}$ and $\mathbb{R}^{p}$ is $\mathcal{C}^{\circ}=\{\bm{0}\}$. So, both $\mathcal{C}_{1}$ and $\mathbb{R}^{p}$ produce the null hypothesis $H_{0}:\bm{\mu}=\bm{0}$. However, $T_{\mathcal{C}_{1}}^{*}$ and $T_{\mathbb{R}^{p}}^{*}$ are clearly different for almost all $\bm{\mu}$ in $\mathbb{R}^{p}$: they only coincide if $\bm{\mu}\in\mathcal{C}_{1}$.

As different cones can correspond to the same null hypothesis, we have the freedom to change the cone $\mathcal{C}$ without affecting the null hypothesis. This allows us to pick $\mathcal{C}$ in such a way that power is directed towards a specific alternative of interest. To describe the alternative corresponding to a cone $\mathcal{C}$, let us again consider $T_{\mathcal{C}}^{*}$, the population equivalent of $T_{\mathcal{C}}$. In particular, suppose that the maximizing vector is $\widehat{\bm{\lambda}}\neq\bm{0}$, then

[TABLE]

where I use the notation $\bm{\varSigma}^{\frac{1}{2}}\mathcal{C}=\{\bm{x}\in\mathbb{R}^{p}\ |\ \bm{x}=\bm{\varSigma}^{\frac{1}{2}}\bm{\lambda},\ \bm{\lambda}\in\mathcal{C}\}$. Notice that $T_{\mathcal{C}}^{*}$ equals $T_{\mathbb{R}^{p}}^{*}$ if $\bm{\varSigma}^{-\frac{1}{2}}\bm{\mu}\in\bm{\varSigma}^{\frac{1}{2}}\mathcal{C}^{\emptyset}$ by the Cauchy-Schwarz inequality, or equivalently if $\bm{\mu}\in\bm{\varSigma}\mathcal{C}^{\emptyset}$. If $\bm{\mu}\not\in\bm{\varSigma}\mathcal{C}^{\emptyset}$, then $T_{\mathcal{C}}^{*}\leq T_{\mathbb{R}^{p}}^{*}$. So, $T_{\mathcal{C}}^{*}$ is ‘large’ if $\bm{\mu}\in\bm{\varSigma}\mathcal{C}^{\emptyset}$ and ‘small’ if $\bm{\mu}\not\in\bm{\varSigma}\mathcal{C}^{\emptyset}$. Hence, $T_{\mathcal{C}}$ seems to be a suitable test statistic for the alternative hypothesis

[TABLE]

Unlike the null hypothesis, this alternative hypothesis is not fully determined by $\mathcal{C}$, but also depends on the the typically unknown parameter $\bm{\varSigma}$. As a consequence, the exact parameter subspace towards which power is directed for a test based on $T_{\mathcal{C}}$ may be unknown in practice. As this is inconvenient, I propose to instead use $T_{\mathcal{C}}$ to test $H_{0}:\bm{\mu}\in\mathcal{C}^{\circ}$ against

[TABLE]

The consequence of this approach is a possible distortion of the alternative hypothesis. While this will clearly lead to a loss of power, the resulting test may still be substantially more powerful than a test based on a quadratic statistic. This is demonstrated numerically in the Monte Carlo experiments presented in Section 5. In addition, there exists an important special case where there is no distortion. This special case is treated in the following section.

3.3 Diagonal covariance matrices and scones

There exists a type of cone $\mathcal{C}$ and covariance matrix $\bm{\varSigma}$ for which $\mathcal{C}=\bm{\varSigma}\mathcal{C}$, so that $H_{1}$ and $H_{1}^{*}$ coincide. In particular, suppose that the elements of $\bm{m}$ are uncorrelated, so that $\bm{\varSigma}$ is diagonal. Let $\mathcal{D}$ be a set of vectors that is closed under multiplication by a positive definite diagonal matrix. That is, if $\bm{\lambda}\in\mathcal{D}$, then $\bm{D}\bm{\lambda}\in\mathcal{D}$, where $\bm{D}>0$. Notice here that $\mathcal{D}$ is a cone.555This follows from the observation that multiplication by a scalar $\gamma>0$ is equivalent to pre-multiplication with by diagonal matrix $\gamma\bm{I}$, where $\bm{I}$ is the identity matrix. So, $\mathcal{D}$ is also closed under positive scalar multiplication, which makes it a cone. So, we have indeed found a type of cone and covariance matrix for which $\mathcal{D}=\bm{\varSigma}\mathcal{D}$.

I have been unable to find a name of these types of cones and will henceforth refer to them as scones (sign cone). A scone can be viewed as a union of sub-orthants (i.e. a union of possibly lower dimensional orthants embedded in a higher dimensional space). As pre-multiplication by a diagonal matrix $\bm{D}>0$ is a sign-preserving operation, the sconic parameter subspaces relate directly to hypotheses concerning sign or sparsity restrictions. To provide some more intuitions for scones, the following examples are given.

Example 5**.**

The scones in $\mathbb{R}^{2}$ are the origin, the four half-axes, the four quadrants, and any union of any these objects.

Example 6**.**

The set of $k$-sparse vectors, defined by $\mathcal{D}_{k}=\{\bm{\mu}\ |\ \|\bm{\mu}\|_{0}\leq k\}$ is a scone. To see this, suppose $\bm{\mu}=(\mu_{1},\dots,\mu_{k},0,\dots,0)^{\prime}$, without loss of generality. So $\bm{\mu}\in\mathcal{D}_{k}$. Let $\bm{D}>0$ have diagonal elements $d_{1},\dots,d_{p}$. Then $\|\bm{D}\bm{\mu}\|_{0}=\|(d_{1}\mu_{1},\dots,d_{k}\mu_{k},0,\dots,0)^{\prime}\|_{0}=\|\bm{\mu}\|_{0}$. So $\bm{D}\bm{\mu}\in\mathcal{D}_{k}$.

If $\bm{\varSigma}$ is assumed to be diagonal, it would be sensible to use a diagonal estimator $\bm{S}$. In this case, the computation of $T_{\mathcal{C}}$ can be simplified substantially. In particular, Proposition 2 shows that computing $T_{\mathcal{C}}$ requires solving a minimum distance problem. Such problems are typically easier to solve than linear regression. For example, the following section considers testing against sparse subspaces, for which this problem has a closed-form solution.

Proposition 2**.**

Let $\bm{S}>0$ be a diagonal matrix and let $\mathcal{D}$ be a scone. Suppose that $\widehat{\bm{\lambda}}=\operatorname*{arg\,max}_{\begin{subarray}{c}\bm{\lambda}\in\mathcal{S}^{0}\cap\mathcal{D}\end{subarray}}\bm{m}^{\prime}\bm{\lambda}$ and $\widehat{\bm{\beta}}=\operatorname*{arg\,min}_{\begin{subarray}{c}\bm{\beta}\in\mathcal{D}\end{subarray}}\|\bm{S}^{-\tfrac{1}{2}}\bm{m}-\bm{\beta}\|_{2}^{2}$ are unique optimizers. Then

[TABLE]

if $\widehat{\bm{\beta}}\neq\bm{0}$ and $\widehat{\bm{\lambda}}=\bm{0}$, otherwise.

4 Sparse parameter subspaces

In this section, I illustrate the methodology described in the previous sections by testing against sparse parameter subspaces. For this illustration, I consider the following data generating process

[TABLE]

where $\bm{E}$ is a random $n\times p$ matrix.

A sparse parameter subspace can be described as follows. For a given level of sparsity $k$, the $k$-sparse parameter subspace $\mathcal{D}_{k}=\{\bm{\mu}\in\mathbb{R}^{p}\ |\ \|\bm{\mu}\|_{0}\leq k\}$ consists of all $k$-sparse vectors of length $p$. Recall here that $\|\bm{x}\|_{0}=\sum_{j=1}^{p}1_{\{x_{j}\neq 0\}}$ is the $\ell_{0}$-norm of a vector $\bm{x}$, which counts the number of non-zero elements in the vector. As detailed in Example 6, $\mathcal{D}_{k}$ is a scone, because $\|\bm{x}\|_{0}=\|\bm{D}\bm{x}\|_{0}$ for all diagonal matrices $\bm{D}>0$.

The hypotheses for testing against a $k$-sparse parameter subspace can then be formulated as

[TABLE]

The interpretation of the alternative $H_{1}^{k}$ is that (at most) $k$ elements of the equalities specified by the null hypothesis are violated.

Following the methodology proposed in Section 3, the statistic that we should use for the $k$-sparse alternative is $T_{\mathcal{D}_{k}}$, which will be abbreviated as $T_{k}$ for notational convenience. In light of the data generating process (1), let us specify the statistic by choosing $\bm{m}=\widehat{\bm{\mu}}:=\tfrac{1}{n}\bm{X}^{\prime}\bm{\iota}_{n}$ to be the sample mean and $\bm{S}=\widehat{\bm{\varSigma}}:=\tfrac{1}{n}\bm{X}^{\prime}(\bm{I}_{n}-\bm{P}_{\bm{\iota}_{n}})\bm{X}$ to be the sample covariance matrix.

Following Condition 1, a sufficient existence condition for $T_{k}$ is then given by

Condition 2**.**

$\operatorname*{rank}(\widehat{\bm{\varSigma}})\geq k$.

This condition is typically satisfied if the number of observations $n$ exceeds the hypothesized sparsity level $k$. So, if the hypothesized sparsity level is sufficiently small, is a much weaker condition than the requirement that $\widehat{\bm{\varSigma}}>0$ which requires that the number of observations exceeds the total number of parameters $p$.

Although the maximizing vector $\widehat{\bm{\lambda}}$ in the computation of $T_{k}$ can be viewed as a by-product, it may also be of interest in its own right. In some applications it may be interesting to not just know whether $H_{0}:\bm{\mu}=\bm{0}$ is violated, but also which of the $p$ equalities specified by the null hypothesis are violated. A collection of $k$ potentially violated equalities is provided by the $k$ non-zero elements of $\widehat{\bm{\lambda}}$. The computation of $\widehat{\bm{\lambda}}$ is discussed in the following section.

4.1 Computation: full covariance matrix

For the case that $\bm{m}=\widehat{\bm{\mu}}$ and $\bm{S}=\widehat{\bm{\varSigma}}$, Proposition 3 shows that the computation of $T_{\mathcal{C}}$ reduces to regularized linear regression where the regularization is determined by $\mathcal{C}$. The result is proven in Appendix A.

Proposition 3**.**

Let $\bm{\lambda}^{\prime}\widehat{\bm{\varSigma}}\bm{\lambda}>0$ for all $\bm{\lambda}\in\mathcal{C}^{\emptyset}$. Let $\widehat{\bm{\beta}}=\operatorname*{arg\,min}_{\begin{subarray}{c}\bm{\beta}\in\mathcal{C}\end{subarray}}\frac{1}{n}\|\bm{\iota}_{n}-\bm{X}\bm{\beta}\|_{2}^{2}$ and $\widehat{\bm{\lambda}}=\operatorname*{arg\,max}_{\begin{subarray}{c}\bm{\lambda}\in\mathcal{S}^{0}\cap\mathcal{C}\end{subarray}}\bm{\mu}^{\prime}\bm{\lambda}$ be unique optimizers. Then

[TABLE]

if $\widehat{\bm{\beta}}\neq\bm{0}$ and $\widehat{\bm{\lambda}}=\bm{0}$, otherwise.

If $\mathcal{C}=\mathcal{D}_{k}$, this result implies that $T_{k}$ can be computed using sparse linear regression or ‘best subset selection’ (BSS), which is defined as follows

[TABLE]

While this optimization problem was long deemed computationally infeasible for $p\gtrsim 40$, Bertsimas et al. (2016) have recently shown that it can be solved for problems of practical size within reasonable time by using gradient descent methods and mixed-integer optimization solvers. In particular, they solve BSS with $n$ of order $10^{3}$ and $p$ of order $10^{2}$ in minutes. If $p$ and $n$ are very large or if re-sampling is used to construct a critical value, then this may still be costly in terms of computation to conduct a test. However, promising work by Hazimeh and Mazumder (2018) shows that problems of order $p=10^{6}$ can be approximated in seconds. An alternative approach is to approximate the solution of (2) using greedy methods such as forward stepwise selection (see e.g. Hastie et al. 2009, 2017).

4.2 Computation: diagonal covariance matrix

Let us now also consider the computation of the statistic for the case that $\bm{S}$ is a positive definite diagonal estimator. Popular examples of diagonal estimators are $\operatorname{diag}(\widehat{\bm{\varSigma}})$ or the pooled variance estimator $\operatorname{tr}(\widehat{\bm{\varSigma}}/p)\bm{I}_{p}$. In order to distinguish the statistic for the diagonal covariance estimator, I use the superscripted notation $T_{k}^{d}$. Proposition 2 then shows that if $\bm{S}>0$, $T_{k}^{d}$ can be computed by solving

[TABLE]

It is straightforward to show that $\widehat{\bm{\beta}}=\bm{S}^{-1}\widehat{\bm{\mu}}_{\mathcal{J}_{k}}$, where $\mathcal{J}_{k}$ contains the largest $k$ elements of $\bm{S}^{-1}\widehat{\bm{\mu}}$.666See e.g. Proposition 3 of Bertsimas et al. (2016) for a formal proof. This leads to the closed form

[TABLE]

If $\bm{S}=\text{diag}(\widehat{\bm{\varSigma}})$, the $T_{k}^{d}$ statistic is closely related to threshold-type statistics (see e.g. Fan, 1996; Zhong et al., 2013), which include the screening statistic proposed by Fan et al. (2015). They define the screening statistic as

[TABLE]

where $\mathcal{J}=\{j\in\{1,\dots,p\}\ |\ |\widehat{\mu}_{j}|>\widehat{\sigma}_{j}\delta\}$, for a given threshold value $\delta$. Here, Fan et al. (2015) choose $\delta$ to grow sufficiently fast so that using 0 as critical value results in a test with asymptotic size 0. Notice that if $\delta$ is instead chosen such that $|\mathcal{J}|=k$, then $J_{0}$ is a monotone function of $T_{k}^{d}$: $J_{0}=\sqrt{p}(T_{k}^{d})^{2}$. Hence, the difference is that a threshold-type statistic implicitly defines the sparsity level, while the $T_{k}^{d}$ explicitly specifies the sparsity level of the alternative. The latter option seems more intuitive, but there may be applications for which a threshold-type specification of the sparsity level is desirable.

4.3 Lasso

Another popular tool to obtain sparse solutions is Lasso (Tibshirani, 1996), which is linear regression under an $\ell_{1}$-norm restriction. Unlike $\mathcal{D}_{k}$, the Lasso constraint set $\mathcal{B}_{t}=\{\bm{\lambda}\ |\ \|\bm{\lambda}\|_{1}\leq t\}$, where $t\geq 0$, is not a scone nor even a cone. However, it is possible to transform a non-cone restriction to a cone restriction, which I will demonstrate in this section.

Let $\Lambda_{t}=\mathcal{V}\cap\mathcal{B}_{t}$ be the intersection of the Lasso constraint set and $\mathcal{V}$, for some $t\geq 0$. Then we can construct the cone $\mathcal{C}_{\Lambda_{t}}=\{\bm{\lambda}\ |\ \gamma\bm{\lambda}\in\Lambda_{t},\exists\gamma>0\}\cup\{\bm{0}\}$ that consists of all the scalar multiples of $\Lambda_{t}$ and the origin. This leads to the alternative

[TABLE]

Notice that if $\bm{\varSigma}=\bm{I}_{p}$, then this alternative coincides with the 1-sparse alternative when $t=1$, and the Wald statistic if $t\geq\sqrt{p}$. A visual illustration of the two-dimensional case is given in Figure 2, for $t=1.2$. This figure shows that the alternative corresponding to the Lasso constraint is set of cones that are centered around the axes, and whose angles are regulated by the parameter $t$. Therefore, the alternative induced by Lasso can be viewed as a near-sparse alternative. The interpretation of near-sparse alternatives extends to higher dimensions.

By Proposition 3, the computation of the test statistic $T_{\mathcal{C}_{\Lambda_{t}}}$ reduces to Lasso regression

[TABLE]

so that $\widehat{\bm{\lambda}}=\widehat{\bm{\beta}}/\sqrt{\widehat{\bm{\beta}}^{\prime}\widehat{\bm{\varSigma}}\widehat{\bm{\beta}}}$.777Unfortunately, as of writing this manuscript, all existing efficient implementations of Lasso that I could find fail for a constant dependent variable. This has prohibited the inclusion of the Lasso based test in the Monte Carlo experiments. The authors of several prominent Lasso implementations have been made aware of this issue.

4.4 Critical values

As the analytical or even the asymptotic derivation of a critical value for $T_{\mathcal{C}}$ is non-trivial, I instead choose to compute a critical value using a re-sampling method. In particular, I use a reflection-based randomization test. Under a symmetry assumption on $\bm{E}$, this leads to an exact critical value for tests with null hypothesis $H_{0}:\bm{\mu}=\bm{0}$, so that size is controlled even in finite samples.888Koning and Bekker (2019) apply a reflection-based randomization test to a composite null hypothesis. Although they are unable to formally show that such a test maintains size for all parameter values under the null, they do provide simulation results that support this conjecture. If this symmetry assumption holds asymptotically, Canay et al. (2017) have shown that such a randomization test has asymptotic size control. The remainder of this section briefly describes the construction of such a critical value (see e.g. Ch. 15 of Lehmann and Romano (2006) for a more general discussion).

Let $\mathcal{R}$ contain all $n\times n$ reflection matrices, which are diagonal matrices with diagonal elements in $\{-1,1\}$. Then $\mathcal{R}$ constitutes the reflection group of order $N=2^{n}$. The assumption that permits the construction of an exact test is that $\bm{E}$ and $\bm{R}\bm{E}$ share the same distribution, for all $\bm{R}\in\mathcal{R}$. This is equivalent to assuming that the distribution of $\bm{E}$ is reflection symmetric.

In order to perform a test, let $\mathcal{R}^{M}:=\{\bm{I}_{n},\bm{R}_{2},\dots,\bm{R}_{M}\}\subseteq\mathcal{R}$, where $\bm{R}_{i}$ is drawn without replacement from $\mathcal{R}\setminus\{\bm{I}_{n}\}$, $i=2,\dots,M$. Here, $M\leq N$ is the number of re-samples. Denote the set containing the reflection transformed data by $\mathcal{R}^{M}_{\bm{X}}=\{\bm{X},\bm{R}_{2}\bm{X},\dots,\bm{R}_{M}\bm{X}\}$. Let $T_{\mathcal{C}}(\mathcal{R}^{M}_{\bm{X}})=\{T_{\mathcal{C}}(\bm{X}),T_{\mathcal{C}}(\bm{R}_{2}\bm{X}),\dots,T_{\mathcal{C}}(\bm{R}_{M}\bm{X})\}$ be the set containing test statistic of interest computed on each of the reflection transformed data sets. Notice that under $H_{0}:\bm{\mu}=\bm{0}$ we have $\bm{X}=\bm{E}$, so that each of the elements of $T_{\mathcal{C}}(\mathcal{R}_{\bm{X}}^{M})$ follow the same distribution. So, under $H_{0}$, the probability that $T_{\mathcal{C}}(\bm{X})$ is larger than the $1-\alpha$ quantile $c_{\alpha}$ among the elements of $T_{\mathcal{C}}(\mathcal{R}_{\bm{X}}^{M})$ is $\alpha$. Therefore, a test with critical value $c_{\alpha}$ that rejects if $T_{\mathcal{C}}(\bm{X})>c_{\alpha}$ has significance level $\alpha$.

5 Simulation experiments

In this section, the Monte Carlo simulation experiments are presented. In particular, I consider testing against sparse alternative, where the level of sparsity is varied. The goal of the experiments is to analyze the size and power properties of the $T_{\mathcal{C}}$ test, and to compare it with the power enhancement approach of Fan et al. (2015). For this purpose, I will vary the sample size, the number of parameters, the underlying covariance matrix and the level of sparsity in the parameter vector. Code to replicate the experiments will be made available at https://github.com/nickwkoning.

5.1 Data generation

I generate the $n\times p$ data matrix $\bm{X}=\bm{\iota}_{n}\bm{\mu}^{\prime}+\bm{E}\bm{A}$, where the elements of $\bm{E}$ are independently drawn from a standard normal distribution, and $\bm{A}^{\prime}\bm{A}=\bm{\varSigma}$, where $\bm{\varSigma}$ has off-diagonal elements $\rho\in\{0,\ .5,\ .7\}$, $i\neq j$ and diagonal elements equal to 1. For the number of observations and parameters, I consider $n\in\{30,\ 250\}$ and $p\in\{100,\ 300,\ 500\}$, respectively.

The parameter vector of interest $\bm{\mu}$ has elements $\mu_{j}=b_{n,s}>0$ for $1\leq j\leq s$ and $\mu_{j}=0$, if $s<j\leq p$. So, $\bm{\mu}$ is $s$-sparse. The values of $b_{n,s}$ are chosen to ensure that all tests that control size have rejection rates smaller than 1 at the significance level $\alpha=0.05$. In particular, I use

[TABLE]

5.2 Tests

On the data described above, I test the null hypothesis $H_{0}:\bm{\mu}=\bm{0}$ against the $s$-sparse alternative $H_{1}^{s}:\bm{\mu}\in\mathcal{D}_{s}$. As this sparsity level may not be known in practice, I also consider a possible over- and underspecification of $s$ by testing against $H_{1}^{1}:\bm{\mu}\in\mathcal{D}_{1}$, if $s=20$, and against $H_{1}^{20}:\bm{\mu}\in\mathcal{D}_{20}$, if $s=1$.

In particular, I use the $T_{k}$ statistic where $k\in\{1,20\}$. As estimators for $\bm{\mu}$ and $\bm{\varSigma}$, I choose the sample mean $\bm{m}=\bm{X}^{\prime}\bm{\iota}_{n}/n$, and sample covariance $\bm{S}=\widehat{\bm{\varSigma}}:=\bm{X}^{\prime}(\bm{I}_{n}-\bm{P}_{\bm{\iota}_{n}})\bm{X}/n$. In addition, I also consider the $T_{k}^{d}$ statistic with $k\in\{1,20\}$, where $\bm{S}=\text{diag}(\widehat{\bm{\varSigma}})$. Note here that the $T_{1}^{d}$ statistic and $T_{1}$ statistic coincide.

The $T_{k}$ statistic is computed using the methodology proposed by Hazimeh and Mazumder (2018). To construct a critical value for $T_{k}$ and $T_{k}^{d}$, I use reflection based randomization as described in Section 4.4. Note that this will yield an exact test, as the normal distribution is reflection symmetric. The number of reflection re-samples used is 1000. Performing a single test with the $T_{20}$ statistic on the largest setting with $n=250$ and $p=500$ takes approximately 45 seconds on a standard 2017 edition 13 inch Macbook Pro.

For the comparison to the power enhancement technique, an initial test and power enhancement test must be selected. As initial test I use the standard Wald test, and as statistic for the enhancement test I use the same screening statistic as used by Fan et al. (2015).999One may argue that a comparison could instead be made to the ‘feasible Wald test’ proposed as initial test by Fan et al. (2015). However, this test relies on an additional sparsity assumption on $\bm{\varSigma}$ which would cloud the comparison. This screening statistic is given by

[TABLE]

with $\mathcal{J}=\{j\in\{1,\dots,p\}\ |\ |m_{j}|>\widehat{\sigma}_{j}\delta_{n,p}\}$, where $\widehat{\sigma}_{j}^{2}$ are the diagonal elements of $\widehat{\bm{\varSigma}}$, and $\delta_{n,p}=\log\log n\sqrt{\log p}$.101010The code provided the online supplementary material of Fan et al. (2015) suggests that $1.06\delta_{n,p}$, $\sqrt{1.5}\delta_{n,p}$ and $.9\delta_{n,p}$ were used in their numerical experiments, instead of $\delta_{n,p}$. It is unclear where these constants come from, as they are not mentioned in the paper. I therefore choose to simply use $\delta_{n,p}$, instead. As critical value for the Wald statistic I use $pn/(n-p)$ times the $1-\alpha$ quantile of the $F$ distribution with $p$ and $n-p$ degrees of freedom (also known as a Hotelling $T^{2}$ test). If $p>n$, the initial test is not defined and I reject with probability $\alpha$. As critical value for the enhancement test I use 0. The combined test then rejects if either the initial test or enhancement test rejects.111111The formulation of the power enhancement technique here differs slightly from the one described by Fan et al. (2015) and is of the form analyzed by Kock and Preinerstorfer (2019). This formulation permits the use of the power enhancement approach even if the test statistic for the initial test does not exist. The simulation experiments are also conducted on the initial test to infer the contribution of the constituent tests to the performance of the power enhancement technique.

For each combination of setting and test, I perform 1000 repetitions and record the proportion of rejections. The results are reported in Tables 2, 3 and 4.

5.3 Results

Table 2 reports the rejection rates of the tests on data that was generated under the null hypothesis. As these rejection rates are used to infer the size of the tests, they should ideally be close to the nominal significance level $\alpha=0.05$. As expected, the table shows that the $T_{\mathcal{C}}$ tests and the Wald test all have good size control under $H_{0}$.

In contrast, the power enhancement test suffers from a small size distortion in the large sample ($n=250$). Furthermore, as the power enhancement test is based on an asymptotic critical value it fails to control size in the small sample ($n=30$). The rejection rates for the power enhanced test for $n=30$ will therefore be ignored in the remainder without mention.

Table 3 contains the rejection rates under the alternative where $s=1$, so that only a single element of $\bm{\mu}$ is unequal to zero. As this data is generated under the alternative, the rejection rates are used to infer the power of the test, which should be as large as possible. Overall, the $T_{1}$ test outperforms all other tests. For the large sample ($n=250$), the power enhanced test performs second best and delivers a large power improvement compared to the Wald test. Despite the mis-specification of the sparsity level, the $T_{20}$ test performs reasonably well if $n=250$, substantially outperforming the Wald test. The $T_{20}$ test performs especially well under large correlations. As expected, the $T_{20}^{d}$ performs better than the $T_{20}$ test under no correlations, but loses power as the correlations increase.

Table 4 presents the results for $s=20$, which are more diverse than the results in Table 2 and 3. The $T_{20}^{d}$ test outperforms the other tests if $\rho=0$ as the sparsity level is now correctly specified. If $\rho\neq 0$ and $n=30$, the $T_{1}$ test outperforms the $T_{20}$ test even though $s=20$. This may be a consequence of the low rank of $\widehat{\bm{\varSigma}}$, so that Condition 2 is barely satisfied. For $n=250$ and $\rho\neq 0$, the power enhancement test performs well in the low-dimensional setting $p=100$. This performance seems almost entirely carried by the initial Wald test, which performs well as $s$ is large compared to $p$. In the settings where $p>n$, the $T_{20}$ test substantially outperforms the other tests.

From the results described above, the following intuitions are drawn.

1. $\bullet$

The $T_{k}^{d}$ test performs well if $\bm{\varSigma}$ is diagonal and $k$ is close to $s$. It loses power as $k$ is further from $s$ or $\bm{\varSigma}$ is further from diagonal. 2. $\bullet$

If $\bm{\varSigma}$ is not diagonal, the $T_{k}$ test performs well if $k$ is close to the true sparsity level $s$, and $n$ is sufficiently large compared to $k$. 3. $\bullet$

The power enhancement test trades a small size distortion for a possibly large power gain. 4. $\bullet$

The power enhancement test is competitive with the $T_{s}$ test if $s$ is small, but loses power compared to the $T_{s}$ test if $s$ is somewhat larger.

6 Appendix A

In order to present the proofs of Proposition 1, I first provide the following simple lemma, where the notation $\gamma\mathcal{A}$ is used to mean the set of containing the elements of $\mathcal{A}\subseteq\mathbb{R}^{p}$ scalar multiplied by $\gamma$.

Lemma 1**.**

Let $\mathcal{C}$ be a cone and $\gamma>0$. If Condition 1 holds, then

[TABLE]

Proof.

As Condition 1 holds, a maximizing argument exists. I find

[TABLE]

∎

Proposition 1**.**

Let $\bm{\lambda}^{\prime}\bm{S}\bm{\lambda}>0$ for all $\bm{\lambda}\in\mathcal{C}^{\emptyset}$. Let $\widehat{\bm{\beta}}=\operatorname*{arg\,min}_{\bm{\beta}\in\mathcal{C}}1-2\bm{m}^{\prime}\bm{\beta}+\bm{\beta}^{\prime}\bm{S}\bm{\beta}$ and $\widehat{\bm{\lambda}}=\operatorname*{arg\,max}_{\begin{subarray}{c}\bm{\lambda}\in\mathcal{S}^{0}\cap\mathcal{C}\end{subarray}}\bm{m}^{\prime}\bm{\lambda}$ be unique optimizers. Then

[TABLE]

if $\widehat{\bm{\beta}}\neq\bm{0}$ and $\widehat{\bm{\lambda}}=\bm{0}$, otherwise.

Proof.

I will consider two cases: $\widehat{\bm{\beta}}\neq\bm{0}$ and $\widehat{\bm{\beta}}=\bm{0}$.

Case $\widehat{\bm{\beta}}\neq\bm{0}$

I find

[TABLE]

where the second equality follows from Lemma 1, the third equality from the fact that $\bm{\lambda}^{\prime}\bm{S}\bm{\lambda}$ is constant due to the constraint, and the final equality from the definition of $\widehat{\bm{\beta}}$.

Case $\widehat{\bm{\beta}}=\bm{0}$

I will prove that $\widehat{\bm{\lambda}}=\bm{0}$ by contradiction. Suppose that $\widehat{\bm{\lambda}}\in\mathcal{C}^{\emptyset}$ is arbitrarily given. Then $\widehat{\bm{\lambda}}^{\prime}\bm{S}\widehat{\bm{\lambda}}-2\bm{m}^{\prime}\widehat{\bm{\lambda}}>0$, as $\widehat{\bm{\beta}}=\bm{0}$ is a unique minimizer. So $\tfrac{1}{2}>\bm{m}^{\prime}\widehat{\bm{\lambda}}/\widehat{\bm{\lambda}}^{\prime}\bm{S}\widehat{\bm{\lambda}}$, because we assumed that $\bm{\lambda}^{\prime}\bm{S}\bm{\lambda}>0$, for all $\bm{\lambda}\in\mathcal{C}^{\emptyset}$. As $\widehat{\bm{\lambda}}\in\mathcal{C}^{\emptyset}$, we have that $\gamma\widehat{\bm{\lambda}}\in\mathcal{C}^{\emptyset}$ for all $\gamma>0$. Then $\lim\limits_{\gamma\to 0^{+}}\bm{m}^{\prime}(\gamma\widetilde{\bm{\lambda}})/(\gamma\widetilde{\bm{\lambda}})^{\prime}\bm{S}(\gamma\widetilde{\bm{\lambda}})=\lim\limits_{\gamma\to 0^{+}}\tfrac{1}{\gamma}\bm{m}^{\prime}\widehat{\bm{\lambda}}/\widehat{\bm{\lambda}}^{\prime}\bm{S}\widehat{\bm{\lambda}}<1/2$, which implies $\bm{m}^{\prime}\widehat{\bm{\lambda}}\leq 0$. This contradicts the assumption that $\widehat{\bm{\lambda}}$ is a unique minimizer. Hence, $\widehat{\bm{\lambda}}=\bm{0}$. ∎

Proposition 2**.**

Let $\bm{S}>0$ be a diagonal matrix and let $\mathcal{D}$ be a scone. Suppose that $\widehat{\bm{\lambda}}=\operatorname*{arg\,max}_{\begin{subarray}{c}\bm{\lambda}\in\mathcal{S}^{0}\cap\mathcal{D}\end{subarray}}\bm{m}^{\prime}\bm{\lambda}$ and $\widehat{\bm{\beta}}=\operatorname*{arg\,min}_{\begin{subarray}{c}\bm{\beta}\in\mathcal{D}\end{subarray}}\|\bm{S}^{-\tfrac{1}{2}}\bm{m}-\bm{\beta}\|_{2}^{2}$ are unique optimizers. Then

[TABLE]

if $\widehat{\bm{\beta}}\neq\bm{0}$ and $\widehat{\bm{\lambda}}=\bm{0}$, otherwise.

Proof.

I will consider two cases: $\widehat{\bm{\beta}}\neq\bm{0}$ and $\widehat{\bm{\beta}}=\bm{0}$.

Case $\widehat{\bm{\beta}}\neq\bm{0}$

Using the substitution $\bm{\beta}=\bm{S}^{\frac{1}{2}}\bm{\lambda}$ and $\widehat{\bm{m}}=\bm{S}^{-\frac{1}{2}}\bm{m}$ yields

[TABLE]

where $\bm{\beta}\in\mathcal{D}_{\bm{\varLambda}}$ if $\bm{\lambda}\in\mathcal{D}_{\bm{\varLambda}}$, as $\bm{S}$ is diagonal and positive-definite. Define $\widehat{\bm{\beta}}=\operatorname*{arg\,min}_{\begin{subarray}{c}\bm{\beta}\in\mathcal{D}_{\bm{\varLambda}}\end{subarray}}\|\bm{\beta}-\widehat{\bm{m}}\|_{2}^{2}$. From Lemma 1 it follows that

[TABLE]

Case $\widehat{\bm{\beta}}=\bm{0}$.

This case is analogous to the case that $\widehat{\bm{\beta}}=\bm{0}$ in the proof of Proposition 1. ∎

Proposition 3**.**

Let $\bm{\lambda}^{\prime}\widehat{\bm{\varSigma}}\bm{\lambda}>0$ for all $\bm{\lambda}\in\mathcal{C}^{\emptyset}$. Let $\widehat{\bm{\beta}}=\operatorname*{arg\,min}_{\begin{subarray}{c}\bm{\beta}\in\mathcal{C}\end{subarray}}\frac{1}{n}\|\bm{\iota}_{n}-\bm{X}\bm{\beta}\|_{2}^{2}$ and $\widehat{\bm{\lambda}}=\operatorname*{arg\,max}_{\begin{subarray}{c}\bm{\lambda}\in(\mathcal{S}\cap\mathcal{C})\cup\{\bm{0}\}\end{subarray}}\bm{\mu}^{\prime}\bm{\lambda}$ be unique optimizers. Then

[TABLE]

if $\widehat{\bm{\beta}}\neq\bm{0}$ and $\widehat{\bm{\lambda}}=\bm{0}$, otherwise.

Proof.

Let $\widehat{\bm{G}}=\bm{X}^{\prime}\bm{X}/n$ denote the Gramian matrix. I will consider two cases.

**Case ** $\widehat{\bm{\beta}}\neq\bm{0}$

Writing out the $\ell_{2}$-norm and using Proposition 1 yields

[TABLE]

where I use the fact that $\bm{\lambda}^{\prime}\widehat{\bm{\varSigma}}\bm{\lambda}:=\bm{\lambda}^{\prime}\widehat{\bm{G}}\bm{\lambda}-(\widehat{\bm{\mu}}^{\prime}\bm{\lambda})^{2}>0$ for all $\bm{\lambda}\in\mathcal{C}^{\emptyset}$, implies that $\bm{\lambda}^{\prime}\widehat{\bm{G}}\bm{\lambda}>0$ for all $\bm{\lambda}\in\mathcal{C}^{\emptyset}$. By Lemma 1

[TABLE]

Combining this and applying some straightforward algebra yields

[TABLE]

Case $\widehat{\bm{\beta}}=\bm{0}$

This case is analogous to the case that $\widehat{\bm{\beta}}=\bm{0}$ in the proof of Proposition 1. ∎