Independence Properties of the Truncated Multivariate Elliptical Distributions
Michael Levine, Donald Richards, and Jianxi Su

TL;DR
This paper characterizes the independence properties of truncated multivariate elliptical distributions, especially the truncated multivariate normal, and applies these findings to test independence in educational data.
Contribution
It provides a novel characterization of independence in truncated multivariate elliptical distributions, extending known results for the normal case.
Findings
Mutual independence of sub-vectors implies the joint distribution is truncated multivariate normal.
The paper verifies regularity conditions for applying Wilks' theorem in a practical test.
Application to educational data demonstrates the usefulness of the independence criterion.
Abstract
Truncated multivariate distributions arise extensively in econometric modelling when non-negative random variables are intrinsic to the data-generation process. More broadly, truncated multivariate distributions have appeared in censored and truncated regression models, simultaneous equations modelling, multivariate regression, and applications going back to the now-classic papers of Amemiya (1974) and Heckman (1976). In some applications of truncated multivariate distributions, there arises the problem of characterizing the distribution through correlation and independence properties of sub-vectors. In this paper, we characterize the truncated multivariate normal random vectors for which two complementary sub-vectors are mutually independent. Further, we characterize the multivariate truncated elliptical distributions, proving that if two complementary sub-vectors are mutually…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsStatistical Distribution Estimation and Applications · Advanced Statistical Methods and Models · Statistical Methods and Bayesian Inference
Independence Properties of the Truncated Multivariate Elliptical Distributions
Michael Levine, Donald Richards, and Jianxi Su Department of Statistics, Purdue University, West Lafayette, IN 47907, U.S.A.Department of Statistics, Pennsylvania State University, University Park, PA 16802, U.S.A.Department of Statistics, Purdue University, West Lafayette, IN 47907, U.S.A. ∗Corresponding author; e-mail address: [email protected]
Abstract
Truncated multivariate distributions arise extensively in econometric modelling, when non-negative random variables are intrinsic to the data-generation process. and more broadly in censored and truncated regression models, simultaneous equations modelling, multivariate regression, and other areas. In some applications, there arises the problem of characterizing truncated multivariate distributions through correlation and independence properties of sub-vectors. In this paper, we characterize the truncated multivariate normal random vectors for which two complementary sub-vectors are mutually independent. Further, we characterize the multivariate truncated elliptical distributions, proving that if two complementary sub-vectors are mutually independent then the distribution of the joint vector is truncated multivariate normal, as is the distribution of each sub-vector. As an application, we apply the independence criterion to test the hypothesis of independence of the entrance examination scores and subsequent course averages achieved by a sample of university students; to do so, we verify the regularity conditions underpinning a classical theorem of Wilks on the asymptotic null distribution of the likelihood ratio test statistic.
Key words and phrases. Truncated elliptical distributions, multivariate normal distributions, correlation, independence
2010 Mathematics Subject Classification. Primary: 62H20, 60E05. Secondary: 62E10.
Running head: Truncated elliptical distributions.
1 Introduction
The truncated multivariate normal distributions are a family of distributions that have appeared in simultaneous equations modelling and multivariate regression [2], economics [11], econometric models for auction theory [14], and other areas. Consequently, there exists a wide literature on the properties of these distributions.
To define the truncated multivariate normal distributions, we recall the component-wise partial ordering on -dimensional Euclidean space, : For column vectors and in we write if for all .
Let and let be a positive definite matrix. For , we say that the random vector has a truncated multivariate normal distribution, with truncation point , if the probability density function of is
[TABLE]
where , the normalizing constant, is given by
[TABLE]
We write whenever has the density function (1.1). Further, we denote the usual (untruncated) multivariate normal distribution by .
Suppose that , , and are partitioned into sub-vectors,
[TABLE]
where , , and each are of dimension , , with . Further, we partition so that
[TABLE]
where is of order , for . In a study of the correlation and independence properties of sub-vectors of truncated distributions, we show in Section 2 that the uncorrelatedness of and cannot be characterized by the condition that . Going beyond the study of the correlation properties of , we prove in Section 3 that the condition is necessary and sufficient for and to be mutually independent; in particular, no restrictions are required on or .
More general than the truncated multivariate normal distributions are their elliptical counterparts. For and in , and a positive definite matrix , a random vector is said to have a truncated elliptical distribution, with truncation point , if its probability density function is of the form
[TABLE]
for a non-constant generator . We write with the untruncated counterpart being denoted by . Examples of truncated elliptically contoured distributions are the truncated multivariate Student’s -distributions [12, 16]. We prove in Section 4 that if then, under certain regularity conditions on the generator , a necessary and sufficient condition that and be independent is that . Here again, no conditions are required on or ; moreover, we verify that the stated regularity conditions on are mild since they hold for many familiar elliptical distributions.
In Section 5, we consider for illustrative purposes an application of the criterion derived in Section 3 to testing the hypothesis of independence of the sub-vectors and . We obtain from the classical theorem of Wilks [13] the asymptotic null distribution of the likelihood ratio test statistic, and we provide an application to a data set given by Cohen [9] on the entrance examination scores and subsequent course averages achieved by a large sample of university students.
2 Correlation properties of truncated elliptical distributions
In this section we show, first, that the correlation structure of a multivariate elliptical distribution does not describe the correlation structure of its truncated version. More precisely, even if a particular multivariate elliptical distribution possesses an identity correlation matrix, this fact is not equivalent to the lack of correlation between components of the truncated version of that multivariate elliptical distribution.
We will demonstrate our claim using the bivariate case. Starting with elliptically distributed random variables , set
[TABLE]
without loss of generality, where . Let be the version of that is truncated at . For simplicity, consider the case in which , so that We will now show that uncorrelatedness between and is not equivalent to .
At the outset, let us recall from [7] a stochastic representation for elliptically distributed random variables:
[TABLE]
where is distributed uniformly over the unit circle, and the generating random variable has the density function . Define
[TABLE]
then, , and
[TABLE]
To calculate these conditional expectations, we transform to polar coordinates,
[TABLE]
where the random variable is uniformly distributed on the interval . Letting , we obtain
[TABLE]
Similarly,
[TABLE]
and
[TABLE]
In summary, we have obtained
[TABLE]
Note that implies . Hence, and .
We remark that uncorrelatedness cannot be characterized for all elliptical truncated distributions through the condition . Consider, for instance, the truncated bivariate Student’s -distribution with degrees-of-freedom , where the associated generating variable has the density function that is proportional to , ; this density corresponds to the generalized beta distribution of the second kind [17]. It is straightforward to deduce that
[TABLE]
. Noting that the gamma function is strictly log-convex [4], we have
[TABLE]
, equivalently . By Equation (2.1), ; hence, for the truncated bivariate Student’s -distributions with truncation points equal to the means, the condition implies that and are positively correlated.
We remark that for the above example, uncorrelatedness holds in a limiting sense as ; in that case, and hence . This limiting case corresponds to the truncated bivariate normal distributions, which we treat in the next section.
On the other hand, for given , we can apply Equation (2.1) to construct a plethora of truncated elliptical distributions that are uncorrelated. For the sake of illustration, suppose that ; then and
[TABLE]
Therefore, for any truncated elliptical distributions whose generating variable satisfies
[TABLE]
the variables and are uncorrelated. For example, if follows a gamma distribution with shape parameter and any positive scale parameter, then Equation (2.2) can be satisfied.
We have now shown that even in the bivariate case and for the special case in which the truncation vector equals the mean , the truncated elliptical distributions do not inherit the correlation property of the untruncated elliptical distributions. On the one hand, it is possible that can lead to positively correlated and , as we have seen from the example on the truncated Student’s -distributions. On the other hand, there exist elliptical distributions with such that the components of their truncated versions are uncorrelated.
3 The multivariate normal case
Throughout the rest of the paper, we denote by any zero matrix or vector, irrespective of the dimension. In this section, we prove that the independence property of multivariate normal distributions can be carried over to their truncated counterparts.
Theorem 3.1**.**
Suppose that the random vector is decomposed as in (1.2). Then and are mutually independent if and only if .
We remark that this result was stated in [15, p. 214]. However, an inspection of the purported proof [15, p. 218] reveals that the ‘if’ part of the result solely was established, so the converse assertion has remained open. Unlike the classical untruncated normal distribution, the matrix is not the covariance matrix of , so it is surprising that the independence of and is characterized by the condition .
Proof of Theorem 3.1. First, we note that
[TABLE]
Now suppose that . Then it is evident from (1.1), (1.3), and (3.1) that the density of reduces to a product of two terms corresponding to the distributions and . Consequently, and are mutually independent, and , .
Conversely, suppose that and are mutually independent. For , it is evident that . Since and are mutually independent if and only if and are mutually independent then we can assume, with no loss of generality, that .
Thus, for , suppose that is independent of . By a well-known quadratic form decomposition (Anderson [3, p. 638]), we have
[TABLE]
where . Applying this decomposition to the density function (1.1), we find that in order to calculate the marginal density of it is necessary to consider the integral
[TABLE]
For fixed , suppose that is a -dimensional multivariate normal random vector with \boldsymbol{V}\sim N_{p_{2}}\big{(}\boldsymbol{\mu}_{2}+{\boldsymbol{\Sigma}}_{21}{\boldsymbol{\Sigma}}_{11}^{-1}(\boldsymbol{w}_{1}-\boldsymbol{\mu}_{1}),{\boldsymbol{\Sigma}}_{22\cdot 1}\big{)}. Then the integral (3.3) equals
[TABLE]
Let ; then,
[TABLE]
Since has the same distribution as then it follows that
[TABLE]
and we denote this probability by \Phi_{p_{2}}\big{(}\boldsymbol{\mu}_{2}+{\boldsymbol{\Sigma}}_{21}{\boldsymbol{\Sigma}}_{11}^{-1}(\boldsymbol{w}_{1}-\boldsymbol{\mu}_{1}),{\boldsymbol{\Sigma}}_{22\cdot 1}\big{)}.
Therefore, the marginal density function of is
[TABLE]
. It now follows from (1.1), (3.4), and the quadratic form decomposition (3.2), that the conditional density function of , given , is
[TABLE]
, .
Since is independent of then is constant in . Therefore,
[TABLE]
, , so we obtain
[TABLE]
, . Cancelling common terms, we obtain
[TABLE]
Note that the left-hand side contains no term in , whereas the right-hand side does. Therefore, for all , the coefficient of on the right-hand side necessarily is the zero vector; this can be proved by taking the logarithm of both sides and then calculating the gradient with respect to .
Hence, . Since this holds for all then we obtain . As and are non-singular, it follows that .
Remark 3.2**.**
We remark that since the condition , which is necessary and sufficient for and to be mutually independent, requires no restrictions on , then the same result holds if we let . Consequently, Theorem 3.1 remains valid if is truncated and is untruncated.
4 The elliptical case
In the elliptical case, as in the normal case, we may assume with no loss of generality, that the truncation point is . Suppose that has a truncated elliptical distribution with density function (1.4). Let
[TABLE]
so the joint p.d.f. of is . In characterizing the distribution of through the independence of and , we will require the following regularity conditions on the generator :
- (R1)
for all , is everywhere differentiable on , and its derivative is continuous. 2. (R2)
The support of , i.e., , is dense in . 3. (R3)
As , either tends to zero or diverges.
We remark that these conditions appear to be mild as almost all of the commonly-used elliptical density functions that are described in [7, Chapter 3] satisfy ((R1))-((R3)), an exception being the Kotz distribution with power parameter in the exponential term equal to
Now we establish as a consequence of Theorem 3.1 a result that, under the regularity conditions ((R1))-((R3)), a truncated multivariate elliptical distribution whose component vectors are independent can only be a truncated multivariate normal distribution.
Corollary 4.1**.**
Suppose that the generator satisfies the regularity conditions ((R1))-((R3)). Then and are independent if and only if has a truncated multivariate normal distribution with .
Proof.
If has a truncated multivariate normal distribution with then we have seen before that and are mutually independent, so we need only show the converse.
By integration, we obtain the marginal density function of as
[TABLE]
and then the conditional density of , given , is
[TABLE]
Note that and are independent if and only if the conditional density function, (4.1), of , given , is constant in . By taking logarithms in (4.1) and then applying the gradient operator , we find that a necessary and sufficient condition for and to be independent is that
[TABLE]
for all , . By (3.2),
[TABLE]
substituting this result in (4.2), we find that a necessary and sufficient condition for independence is
[TABLE]
, . Cancelling on both sides of the latter equation, we obtain
[TABLE]
, .
Let be such that . Evaluating both sides of (4.3) at , we obtain
[TABLE]
equivalently,
[TABLE]
for all , where and is a constant vector.
We also have ; otherwise, the left-hand side of (4.4) is infinite for all , and then it follows that is infinite for all . This implies that is unbounded everywhere, which is not possible since generates a density function.
Suppose that ; then, by (4.5), or for all . If for all then it follows that is a constant function; however, by ((R2)), the support of is dense, therefore cannot generate a density. Also, by construction, , so for all . Therefore, we have shown by contradiction that .
Now suppose that . Since is positive definite then is positive semidefinite and has the same rank as . Since then that rank is at least , so at least one diagonal entry of is positive; without loss of generality, we assume that the first diagonal entry, , is positive. Letting , we obtain
[TABLE]
consequently, as .
By (3.2),
[TABLE]
therefore, as , we obtain where
[TABLE]
the th entry of . Letting in (4.5), we obtain
[TABLE]
By the regularity condition ((R3)), tends to zero or diverges as . If then the right-hand side of Equation (4) tends to zero as , so we obtain , which contradicts the fact that . On the other hand, if diverges as , then the right-hand side of Equation (4) diverges, which contradicts the fact that . Since the assumption that leads in either case to a contradiction then it follows that .
Since then Equation (4.5) reduces to
[TABLE]
equivalently, , hence , for some constants and . Therefore, has a truncated multivariate normal distribution with . ∎
5 Testing the independence of the components of a truncated multivariate normal vector
As an application of our results, we perform a likelihood ratio test for independence between and , the components of a bivariate truncated normal random vector. For , denote by the th element of ; let ; and set . By Theorem 3.1, testing for independence between and is equivalent to testing the null hypothesis, , vs. the alternative hypothesis, . For illustrative purposes, we apply the test to a data set, considered by Cohen [9, p. 192], consisting of the entrance examination scores, , and subsequent course averages, , achieved by university students. The data are viewed as generated randomly from a bivariate truncated normal distribution, with the cutoff value for being , the minimum qualifying score on the entrance examination, and with the cutoff value for being since all course averages are nonnegative, respectively. With these constraints, students were admitted.
Corresponding to , we denote the unrestricted (or alternative) parameter space by ; wherever necessary, we may also denote the respective individual components of by , . The restricted (or null) parameter space, as determined by , then is , and the likelihood ratio test statistic for testing vs. is
[TABLE]
For , we write the joint probability density function of in the form
[TABLE]
, , where
[TABLE]
and
[TABLE]
is the normalizing constant.
For a random sample from the distribution (5.1), the corresponding log-likelihood function can be written, up to additive constants that do not depend on the parameter , as
[TABLE]
The asymptotic null distribution of the likelihood ratio statistic is derived from a classical theorem of Wilks [18] (cf., Casella and Berger [8, pp. 489, 516], Hogg, et al. [13, p. 361]). First, we verify that the regularity conditions underlying Wilks’ theorem are valid for the truncated normal distribution:
The density is identifiable, i.e., if for all then : To prove this result, note that
[TABLE]
then it follows from (5.1) that the truncated bivariate normal distribution is an exponential family with natural (or canonical) sufficient statistic
[TABLE]
and corresponding canonical parameter vector
[TABLE]
It is now evident that the components of the natural sufficient statistic and of the canonical parameter vector are linearly independent over . Further, the exponential family is minimal, meaning that it is five-dimensional and cannot be reduced to a lower-dimensional model. Consequently, by Barndorff-Nielsen [5, pp. 112–113, Lemma 8.1 and Corollary 8.2], the model (5.1) is identifiable.
- 2.
The support of the distribution remains the same for all values of : This condition is clearly satisfied since the density has support , which does not depend on .
- 3.
There exists an open subset such that the “true value” of the parameter is in , and all third-order partial derivatives of with respect to exist for all : This condition is satisfied since is an open subset of and we can construct consisting of the union of sufficiently small open univariate balls around the true value of each of the parameters . Further, the differentiability property follows from (5.1).
- 4.
The integral is twice-differentiable with respect to : According to the usual Leibniz rule, partial derivatives and integrals may be interchanged whenever the same derivatives of the density function are continuous and integrable for all and all , ; see Burkill and Burkill [6, p. 289, Theorem 8.72]. In the case of (5.1), the conditions for Leibniz’ rule follows from the finiteness of the moments of any positive order for that distribution. We also note that Barndorff-Nielsen [5, p. 114, Theorem 8.1] shows that differentiation with respect to , to any order, of the integral is allowed under the integral sign.
- 5.
The information matrix of the density function is positive definite: As shown earlier, is a non-curved minimal exponential model. By a well-known result for exponential families [5, Section 9.3], the covariance matrix of , denoted by , is a full-rank matrix and therefore is positive definite. Since the information matrix is
[TABLE]
then it follows that also is of full rank.
- 6.
All third-order partial derivatives of are bounded by functions of that have finite expectations: By straightforward differentiation with respect to , , and , , we obtain
[TABLE]
where is a polynomial in . Since all polynomial moments of the truncated bivariate normal distribution are finite then for all .
Having shown that the regularity conditions underpinning Wilks’ theorem are satisfied in our setting, we deduce that, under , in distribution as .
To apply to the data of Cohen [9, loc. cit.] the likelihood ratio statistic for testing vs. , we calculated the maximum likelihood estimates of the parameters of the bivariate truncated normal distribution using the R package of Wilhelm and Manjunath [19]; alternatively, the calculations can be done using the procedures described by Cohen [9, pp. 186–190]. We obtained from the R package the estimates,
[TABLE]
The resulting observed value of the test statistic was , and the corresponding P-value was found to be approximately . Consequently, the null hypothesis is rejected at any practical level of significance.
6 Conclusions
We have shown that the mutual independence of the components of a multivariate truncated elliptical distribution is equivalent to subject to additional regularity conditions on the generator function . If these regularity conditions are satisfied then they imply that the underlying distribution is the truncated multivariate normal distribution. These results suggest two problems for future research. The first problem concerns the existence of multivariate truncated elliptical distributions, other than the truncated normal, for which is equivalent to independence of its components. This problem leads naturally to a search for regularity conditions weaker than the ones that we have used in Corollary 4.1. The second direction is to characterize the property of uncorrelatedness for the multivariate truncated elliptical distributions; explicitly, the goal will be to obtain explicit criteria, in terms of the correlation matrix of the underlying multivariate elliptical distribution and its generator function, that are equivalent to zero correlation between components of its truncated analogs, and . We plan to study both of these directions in future research.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1]
- 2[2] Amemiya, T. (1974). Multivariate regression and simultaneous equations models when the dependent variables are truncated normal. Econometrica , 42 , 999-1012.
- 3[3] Anderson, T. W. (2003). An Introduction to Multivariate Statistical Analysis , third edition. Wiley, New York, NY.
- 4[4] Artin, E. (1964). The Gamma Function . Holt-Rinehart-Winston, New York, NY.
- 5[5] Barndorff-Nielsen, O. E. (2014). Information and Exponential Families in Statistical Theory , second printing. Wiley, Chichester.
- 6[6] Burkill, J. C., and Burkill, H. (2002). A Second Course in Mathematical Analysis . Cambridge University Press, New York, NY.
- 7[7] Fang, K.-T., Kotz, S., and Ng, K. W. (1990). Symmetric Multivariate and Related Distributions , Chapman & Hall, New York, NY.
- 8[8] Casella, G., and Berger, R. L. (2002). Statistical Inference , second edition, Duxbury Press, Pacific Grove, CA.
