Parametrising correlation matrices

TL;DR

This paper explores parametrisations of correlation matrices, connecting partial correlations and hyperspherical coordinates, and extends these concepts to complex and quaternion cases, also proposing a distribution for random correlation matrices.

Contribution

It unifies different parametrisations of correlation matrices and extends them to complex and quaternion cases, introducing a distribution on the space of correlation matrices.

Findings

Relation between partial correlations and hyperspherical coordinates.

Extension of parametrisation to complex and quaternion matrices.

Distribution on correlation matrices with density proportional to (det R)^a.

Abstract

Correlation matrices are the sub-class of positive definite real matrices with all entries on the diagonal equal to unity. Earlier work has exhibited a parametrisation of the corresponding Cholesky factorisation in terms of partial correlations, and also in terms of hyperspherical co-ordinates. We show how the two are relating, starting from the definition of the partial correlations in terms of the Schur complement. We extend this to the generalisation of correlation matrices to the cases of complex and quaternion entries. As in the real case, we show how the hyperspherical parametrisation leads naturally to a distribution on the space of correlation matrices $\{R\}$ with probability density function proportional to $( \det R)^a$. For certain $a$, a construction of random correlation matrices realising this distribution is given in terms of rectangular standard Gaussian matrices.

Equations

$$Y=\left[\begin{array}{cccc}{\mathbf{Y}}^{(1)}& {\mathbf{Y}}^{(2)}& \cdots & {\mathbf{Y}}^{(N)}\end{array}\right]$$

$$S=\frac{1}{n-1}{Y}^{\top }Y.$$

$$R={\left[\begin{array}{c}\frac{{\left({\mathbf{Y}}^{(j)}\right)}^{\top }{\mathbf{Y}}^{(k)}}{\Vert {\mathbf{Y}}^{(j)}\Vert \Vert {\mathbf{Y}}^{(k)}\Vert }\end{array}\right]}_{j,k=1}^N=:{\left[\begin{array}{c}{\rho }_{jk}\end{array}\right]}_{j,k=1}^N.$$

$$R={\mathbf{D}}^{\top }{Y}^{\top }Y\mathbf{D},$$

$$\mathbf{D}={\left[\begin{array}{c}\frac{1}{\Vert {\mathbf{Y}}^{(j)}\Vert }\end{array}\right]}_{j=1}^N$$

$$R=L{L}^{\top }.$$

$$\mathrm{vol}\left({\mathcal{R}}_N\right)=\prod _{j=2}^N{2}^{(j-1{)}^2}{\left(B(j/2,j/2)\right)}^{j-1}$$

$$B(a,b)=\frac{\Gamma (a)\Gamma (b)}{\Gamma (a+b)}.$$

$$L=\left[\begin{array}{cccccc}{l}_{11}& 0& 0& \cdots & 0& \\ l_{21}& l_{22}& 0& \cdots & 0& \\ l_{31}& l_{32}& l_{33}& \cdots & 0& \\ ⋮& ⋮& ⋮& \ddots & ⋮& \\ l_{N1}& l_{N2}& l_{N3}& \cdots & l_{NN}& \end{array}\right].$$

$$l_{jj}>0,(j=1,\cdots ,N).$$

$$\sum _{k=1}^j{l}_{jk}^2=1.$$

$$l_{jk}=\{\begin{array}{ll}\cos {\theta }_{jk}\prod _{p=1}^{k-1}\sin {\theta }_{jp}& (1⩽k⩽j-1)\\ \prod _{p=1}^{j-1}\sin {\theta }_{jp}& (j=k),\end{array}$$

$$0<{\theta }_{j,k}<\pi (1⩽k<j\le N).$$

$${\left[\begin{array}{c}{\rho }_{j,k|\{1,\cdots ,p-1\}}\end{array}\right]}_{j,k=p}^N={\left[\begin{array}{c}\frac{\left({\mathbf{Y}}^{(j)}-P_{\{1,\cdots ,p-1\}}^{\perp }{\mathbf{Y}}^{(j)}\right)\cdot \left({\mathbf{Y}}^{(k)}-P_{\{1,\cdots ,p-1\}}^{\perp }{\mathbf{Y}}^{(k)}\right)}{\Vert {\mathbf{Y}}^{(j)}-P_{\{1,\cdots ,p-1\}}^{\perp }{\mathbf{Y}}^{(j)}\Vert \Vert {\mathbf{Y}}^{(k)}-P_{\{1,\cdots ,p-1\}}^{\perp }{\mathbf{Y}}^{(k)}\Vert }\end{array}\right]}_{j,k=p}^N$$

$$R=\left[\begin{array}{cc}{R}_{11}& R_{12}\\ R_{21}& R_{22}\end{array}\right],$$

$$R/R_{11}:={\left[\begin{array}{c}{\left(R/R_{11}\right)}_{j,k}\end{array}\right]}_{j,k=p}^N:=R_{22}-R_{21}{R}_{11}^{-1}{R}_{12}.$$

$${\rho }_{jk|\{1,\cdots ,p-1\}}=\frac{(R/R_{11}{)}_{jk}}{(R/R_{11}{)}_{jj}^{1/2}(R/R_{11}{)}_{kk}^{1/2}}$$

$$l_{j1}$$

$$l_{jk}$$

$$l_{jj}$$

$${\rho }_{jk|\{1,\cdots ,k-1\}}=\cos {\theta }_{jk},1⩽k<j⩽N.$$

$$Y^{\top }Y=\Sigma =\left[\begin{array}{c}{\sigma }_{jk}\end{array}\right]=\left[\begin{array}{cc}{\Sigma }_{\mathbf{p}}& {\Sigma }_{\mathbf{pq}}\\ {\Sigma }_{\mathbf{qp}}& {\Sigma }_{\mathbf{q}}\end{array}\right].$$

$$\Sigma /{\Sigma }_{\mathbf{p}}={\left[\begin{array}{c}{\sigma }_{jk|\{1,\cdots ,p-1\}}\end{array}\right]}_{j,k=p}^N={\Sigma }_{\mathbf{q}}-{\Sigma }_{\mathbf{qp}}{\Sigma }_{\mathbf{p}}^{-1}{\Sigma }_{\mathbf{pq}}.$$

$$\Sigma ={\left[\begin{array}{c}{\sigma }_{jk}\end{array}\right]}_{j,k=1}^N=A{A}^{\top }=\left[\begin{array}{cc}{A}_{\mathbf{p}}& 0_{\mathbf{pq}}\\ A_{\mathbf{qp}}& A_{\mathbf{q}}\end{array}\right]\left[\begin{array}{cc}{A}_{\mathbf{p}}^{\top }& A_{\mathbf{qp}}^{\top }\\ 0_{\mathbf{pq}}^{\top }& A_{\mathbf{q}}^{\top }\end{array}\right],$$

$$\Sigma /{\Sigma }_{\mathbf{p}}=A_{\mathbf{qp}}{A}_{\mathbf{qp}}^{\top }+A_{\mathbf{q}}{A}_{\mathbf{q}}^{\top }-A_{\mathbf{qp}}{A}_{\mathbf{p}}^{\top }(A_{\mathbf{p}}{A}_{\mathbf{p}}^{\top }{)}^{-1}{A}_{\mathbf{p}}{A}_{\mathbf{qp}}^{\top }=A_{\mathbf{q}}{A}_{\mathbf{q}}^{\top }.$$

$${\sigma }_{jk|\{1,\cdots ,p\}}={\sigma }_{jk|\{1,\cdots ,p-1\}}-\frac{{\sigma }_{jp|\{1,\cdots ,p-1\}}{\sigma }_{pk|\{1,\cdots ,p-1\}}}{{\sigma }_{pp|\{1,\cdots ,p-1\}}},p+1⩽j,k⩽N.$$

$$\Sigma =\left[\begin{array}{ccc}{\Sigma }_{\mathbf{p}}& {\left[\begin{array}{c}{\sigma }_{jp}\end{array}\right]}_{j=1}^{p-1}& {\Sigma }_{\mathbf{p}\tilde{\mathbf{q}}}\\ {\left[\begin{array}{c}{\sigma }_{pk}\end{array}\right]}_{k=1}^{p-1}& {\sigma }_{pp}& {\left[\begin{array}{c}{\sigma }_{pk}\end{array}\right]}_{k=p+1}^N\\ {\Sigma }_{\tilde{\mathbf{q}}\mathbf{p}}& {\left[\begin{array}{c}{\sigma }_{jp}\end{array}\right]}_{j=p+1}^N& {\Sigma }_{\tilde{\mathbf{q}}}\end{array}\right],$$

$$A=\left[\begin{array}{ccc}{A}_{\mathbf{p}}& {\left[\begin{array}{c}0\end{array}\right]}_{j=1}^{p-1}& 0_{\mathbf{p}\tilde{\mathbf{q}}}\\ {\left[\begin{array}{c}{A}_{pk}\end{array}\right]}_{k=1}^{p-1}& A_{pp}& {\left[\begin{array}{c}0\end{array}\right]}_{k=p+1}^N\\ A_{\tilde{\mathbf{q}},\mathbf{p}}& A_{\tilde{\mathbf{q}},\{p\}}& A_{\tilde{\mathbf{q}}}\end{array}\right].$$

$$\Sigma /{\Sigma }_{\{1,\cdots ,p-1\}}=\left[\begin{array}{cc}{A}_{pp}^2& A_{pp}{A}_{\tilde{\mathbf{q}},\{p\}}^{\top }\\ A_{pp}{A}_{\tilde{\mathbf{q}},\{p\}}& A_{\tilde{\mathbf{q}},\{p\}}{A}_{\tilde{\mathbf{q}},\{p\}}^{\top }+A_{\tilde{\mathbf{q}}}{A}_{\tilde{\mathbf{q}}}^{\top }\end{array}\right].$$

$$\Sigma /{\Sigma }_{\{1,\cdots ,p\}}=A_{\tilde{\mathbf{q}}}{A}_{\tilde{\mathbf{q}}}^{\top }.$$

$${\sigma }_{jk|\{1,\cdots ,p-1\}}$$

Full text

1 Introduction

In applications of matrices, there are many settings in which the rows and columns have distinct meaning. For example, in a survey of $n$ people, giving a numerical score between [math] and $5$ for their rating of $N$ different movies, there is an $n\times N$ matrix $X$—the data matrix—such that the rows correspond to the people and the columns to the movies. The $k$-th column $\mathbf{X}^{(k)}$ is then the vector of scores given for movie $k$, and its $j$-th entry is the score given by person $j$. Let $\mu_{k}$ denote the average of the scores in column $k$, and let $\mathbf{1}_{n}$ denote the $n\times 1$ vector with all entries equal to 1. The recentred, zero mean score vectors are then specified as $\mathbf{Y}^{(k)}=\mathbf{X}^{(k)}-\mu_{k}\mathbf{1}_{n}$, and the recentred data matrix is

[TABLE]

The sample covariance matrix $S$ is specified in terms of $Y$ as

[TABLE]

Note that $S$ is a $N\times N$ symmetric matrix, and its entry in rows $j$ and column $k$ gives the sample covariance between the scores of movies $j$ and $k$. In the case that the rows and/or columns of $Y$ are drawn from a vector Gaussian distribution with given covariance, (1.2) is referred to a Wishart matrix. For such random matrices, a vast number of theoretical results have been assembled, and applied settings identified, since the pioneering paper [21]; see e.g. [13].

Natural from the viewpoint of data analysis is to further refine (1.2) by forming the sample correlation matrix

[TABLE]

Here, as well as the original score vectors being centred by subtracting their mean, each has been scaled to correspond to a unit vector. One sees immediately that the entries of $R$ are all equal to unity on the diagonal, while on the off diagonal, in accordance with the Cauchy-Schwarz inequality, they all have modulus less than or equal to 1. Moreover, the decomposition

[TABLE]

where

[TABLE]

shows that $R$, like $S$, is positive definite but now with bounded entries $|\rho_{jk}|\leqslant 1$.

This latter feature, although making some aspects of theoretical analysis more difficult (e.g. studies of the eigenvalues) allows for a distinct set of questions to be posed. For example, in the case of correlation matrices the volume of the natural embedding in $\mathbb{R}^{N(N-1)}$—referred to as an elliptope [14] and to be denoted $\mathcal{R}_{N}$—is well defined. Knowledge of this volume allows an answer to the question: if the strictly upper triangular entries of (1.3) are chosen uniformly at random in the range $(-1,1)$, what is the probability that $R$ is a valid correlation matrix (i.e. is positive definite) [6]?

A direct approach to this question requires a parametrisation of the space of correlation matrices. Two such parametrisations are available in the literature, both applying to the lower triangular matrix $L$ in the Cholesky factorisation

[TABLE]

One of these use hyperspherical co-ordinates in $\mathbb{R}^{j}$ to parametrise row $j$ ($j=1,\cdots,N$) [17, 19, 18, 16, 6], and the other makes use of a sequence of partial correlations [10, 12, 9]. The latter method yielded the first direct computation of the volume [10]

[TABLE]

where

[TABLE]

It is only in the last few years that this same formula (in equivalent forms) was derived using the hyperspherical parametrisation [16, 6].

Indirect computations of $\mathrm{vol}\left(\mathcal{R}_{N}\right)$ are also possible. Such a method, giving a formula equivalent to (1.6) actually predates the work [10]—this is due to Wong et al. [22]. As implied by a comment in [16, 2nd paragraph of Introduction] the same result, again deduced indirectly, follows from the still earlier work of Muirhead [13, p.148].

The circumstances just described suggest a number of follow up problems. The most immediate is to relate the hyperspherical and partial correlation parametrisations. To give a satisfactory account on this point, a self contained theory relating to the latter must be developed. Moreover, the hyperspherical parametrisation gives a different viewpoint on known results [13] for the marginal distribution of the elements of (1.3), when chosen uniformly at random, and similarly for the moments of $\det R$.

The literature cited above is restricted to the case of real entries. Complex valued covariance matrices, and thus complex valued correlation matrices, are well motivated from the viewpoint of their application in wireless communication; see for example [20]. Thus, in addition to addressing the above problems when the correlation matrices have real entries, we consider too the case of complex (and quaternion) entries.

2 Cholesky factorisation and parametrisations

Let $R$ be an $N\times N$ positive definite matrix with all diagonal entries equal to unity, as is consistent with (1.3). Introduce the Cholesky factorisation (1.5) with

[TABLE]

Well established theory (see e.g. [8]) gives that this is unique for $R$ positive definite subject to the requirement that

[TABLE]

The fact that the diagonal entries in (1.3) are all equal to unity implies that the sum of the squares of the non- zero entries along each rows $j$ of $L$ is also unity,

[TABLE]

Hence $(l_{j1},l_{j2},\cdots,l_{jj})$ is a point on the sphere $S_{j-1}$. As such it permits the hyperspherical parametrisation (see references noted below (1.5))

[TABLE]

where for $k=1$ the products are to be taken as equal to unity. The requirement (2.2) implies

[TABLE]

Let us now turn our attention to the parametrisation of the entries in (2.1) using partial correlation coefficients. In the setting leading to the definition (1.3) one defines the partial correlation coefficients $\rho_{j,k|\{1,\cdots,p-1\}}$ ($p\leq j,k\leq N$) as the entries of the $(N-p+1)\times(N-p+1)$ matrix

[TABLE]

where $P_{\{1,\cdots,p-1\}}^{\perp}\mathbf{x}$ denotes the orthogonal projection of the vector $\mathbf{x}$ onto the hyperplane spanned by $\mathbf{Y}^{(1)},\cdots,\mathbf{Y}^{(p-1)}$ [1].

The partial correlation coefficients are intimately related to the Schur complement of $R$ [15]. In relation to the latter, partition $R$ according to

[TABLE]

where $R_{21}=R_{12}^{\top}$ is of size $(N-p+1)\times(p-1)$, $R_{11}$ is of size $(p-1)\times(p-1)$ and $R_{22}$ is of size $(N-p+1)\times(N-p+1)$. The Schur complement is the $(N-p+1)\times(N-p+1)$ matrix

[TABLE]

In terms of the Schur complement, the partial correlations can be written [1, 15]

[TABLE]

It is stated in [2, $\S$3.3] that the elements in (2.1) as required for the Cholesky factorisation (1.5) can be writen in terms of a subset of the partial correlations according to

[TABLE]

Comparison with the hyperspherical parametrisation (2.4) shows the simple relationship

[TABLE]

Following ideas [3] we will show how (2.9) can be derived in a way that is consistent with (2.7). With $Y$ given by (1.1), this requires introducing the covariance matrix

[TABLE]

Here $\mathbf{p}=\{1,\cdots,p-1\}$, $\mathbf{q}=\{p,\cdots,N\}$ and $\Sigma_{\mathbf{pq}}$ is the sub-block of $\Sigma$ formed from rows $\mathbf{p}$ and columns $\mathbf{q}$ ($\Sigma_{\mathbf{p}}$ is an abbreviation for $\Sigma_{\mathbf{p}\mathbf{p}}$). As is consistent with (2.6) the Schur complement of $\Sigma$ corresponding to the partition (2.10) is

[TABLE]

The quantities $\sigma_{jk|\{1,\cdots,p-1\}}$ are referred to as partial covariances.

As noted in [3], (2.11) can be written in terms of a suitable block partitioning of the Cholesky factorisation of $\Sigma$,

[TABLE]

where $A$ is a lower triangular matrix with entries on the diagonal nonnegative and $A_{\mathbf{p}},A_{\mathbf{qp}}$ etc are sub-blocks as in the notation (2.10). Thus one sees that

[TABLE]

Using this, a known recursive formula for the partial covariances can be re-derived in a structured, self contained way, which moreover relates to (2.9).

Proposition 1**.**

The partial covariances satisfy the recursive formula [1]

[TABLE]

Proof.

Extend (2.10) by writing

[TABLE]

where $\tilde{\mathbf{q}}=\{p+1,\cdots,N\}$. Similarly, in the Cholesky factorisation (2.12), extend the block decomposition of the lower triangular matrix $A$ by writing

[TABLE]

Substituting the bottom $2\times 2$ block of (2.16) for $A_{\mathbf{q}}$ in (2.13) shows

[TABLE]

In relation to $A_{\tilde{\mathbf{q}}}A_{\tilde{\mathbf{q}}}^{\top}$ as appearing in this expression, we note from (2.13) with $\mathbf{p}$ replaced by $\{1,\cdots,p\}$ and $\mathbf{q}$ by $\tilde{\mathbf{q}}$, that

[TABLE]

Recalling the notation in(2.11) for the partial covariances, we read off from (2.17) that

[TABLE]

Similarly, we read off from (2.18) that

[TABLE]

Eliminating all dependence on the matrix $A$ and its entries in (2.19)–(2.22), (2.23) gives (2.14) ∎

The recurrence (2.14), obtained through different working described as ’tedious’, is given in [1, §2.5.3]. And as noted in this latter reference, it follows immediately from (2.14), and the relation between partial correlations and partial covariances

[TABLE]

(compare (2.7) and (2.11)), that there is a similar recurrence to (2.14) for the partial correlations.

Corollary 2**.**

We have

[TABLE]

Most significant in relation to explaining the parametrisation (2.9) are the relations (2.20) and (2.21). Thus substituting in (2.13), recalling the first equality in (2.11) and making use too of (2.24) gives

[TABLE]

Using now a hyperspherical parametrisation of the lower triangular matrix $A$ by setting $A_{jk}=r_{j}l_{jk},(r_{j}>0,\,\sum_{j=1}^{k}A_{jk}^{2}=r_{j}^{2})$, with $l_{jk}$ given by (2.4), we immediately obtain from (2.25) the parametrisation (2.9).

3 The Jacobian, hyperspherical parametrisation of determinant and some consequences

Let $|J_{\{\rho_{jk}\}\mapsto\{\theta_{jk}\}}|$ denote the Jacobian (absolute value of the determinant of the Jacobian matrix) for the change of variables $\{\rho_{jk}\}\mapsto\{\theta_{jk}\}$ as implied by (1.3), (1.5) and (2.4). It is shown in [16] and [6] that upon ordering the entries $\{\rho_{21},\rho_{31},\rho_{32},\cdots\}$ i.e. reading sequentially along rows of the strictly lower triangular portion of $R$, and similarly ordering the angles, the Jacobian matrix is lower triangular. Its determinant and thus the Jacobian can be read off as equal to

[TABLE]

Note that this is strictly positive for the angles in the range (2.5), and vanishes on the boundary of the range. Earlier, it was shown by Joe [10] (upon adjusting for the different convention by way of projected variables, the details of which were subsequently carried out in [11]) that in terms of the partial correlations as appearing in (2)

[TABLE]

The expressions (3.1) and (3.2) are seem to be consistent with (2.9), upon noting that the latter implies

[TABLE]

We see from (3.1) that $|J_{\{\rho_{jk}\}\mapsto\{\theta_{jk}\}}|$ factorises with respect to the variables $\theta_{jk}$. The same is true for $\det R$. Thus, as follows from (1.5) and (2.4) we have [16]

[TABLE]

Joe [10] had earlier shown that in terms of partial correlations

[TABLE]

as is consistent with (3.3) and (2.9).

It follows from (3.2) and (3.3) that the choice of probability density function on the space of correlation matrices

[TABLE]

where $C_{a,N}$ denotes the normalisation, permits the hyperspherical parametrisation of the corresponding measure

[TABLE]

The explicit formula (3.6) first appeared in [16], however an equivalent formula in terms of partial correlations can be found in [10] (again, upon adjusting for the the different convention by way of projected variables [11]).

A consequence of (3.6) is the evaluation of the normalisation

[TABLE]

where $B(\alpha,\beta)$ is given by (1.7). This is already known from [10, 16]. In fact the probability density (3.5) on the space of correlation matrices, with $a=(n-N-1)/2$, ($n\leqslant N,n\in\mathbb{Z}^{+}$), and the evaluation of the normalisation (3.7) albeit written in a different form, first appeared in the work of Muirhead [13, Th. 5.1.3]. It is shown there that it can be realised by choosing in (1.4) the matrix to be of size $n\times N$ with independent standard Gaussian entries.

Setting $a=0$, it follows that in the case of a uniform distribution

[TABLE]

The working needed to show the equality between this form, which was first given in [16], and the form (1.6) as given in [10], can be found in [16].

We read off from (3.6) that the marginal distribution of $\theta_{j,1}$, ($j\geqslant 2$) is proportional to $(\sin\theta_{j1})^{2a+N-1}$. Since for $k=1$, $\rho_{jk|\{1,\cdots,k-1\}}=\rho_{j1}$, we can then make use of (2.9) to deduce the marginal distribution of any one $\rho_{jk}$, $(j>k)$ when $R$ has distribution (3.5).

Proposition 3**.**

In the above setting we have that the marginal distribution of a single non-diagonal element of $R$ has probability density function

[TABLE]

Remark 4*.*

With $a=(n-N-1)/2$, $(n\geqslant N,n\in\mathbb{Z}^{+})$ this result can be found in [13, §5.1 eq.(5)]

Proposition 5**.**

Let $R$ have distribution (3.5). We have

[TABLE]

and

[TABLE]

where $\Psi(x)$ denotes the digamma function.

Proof.

The formula (3.11) is deduced from (3.10) by differentiating with respect to $s$, and setting $s=0$. The formula (3.10) is immediate from (3.3), (3.6) and the trigonometric form of the beta function

[TABLE]

∎

Remark 6*.*

With $a=(n-N-1)/2$, $(n\geqslant N,n\in\mathbb{Z}^{+})$ a result equivalent to (3.10) can be found in [13, §5.1 eq.(9)]. It is note in this reference that the result (3.10) implies that the limiting $a\rightarrow\infty$ distribution of $-2a\log\det R$ is equal to $\chi^{2}_{N(N-1)/2}$.

4 Random correlation matrices with complex or quaternion entries

A correlation matrix can be constructed out of a data matrix $Y$ with complex entries by replacing $Y^{\top}Y$ in (1.4) by $Y^{\top}Y$. As mentioned in settings in wireless communications engineering . Of less practical interest, but still of theoretical relevance within random matrix theory (see e.g. [7]) is to form correlation matrices out of data matrices with entries having $2\times 2$ block structure

[TABLE]

Such $2\times 2$ matrices form a representation of quaternions, allowing matrices with quaternion entries to be written as certain structured complex matrices of even size.

The theory relating to the hyperspherical parametrisation of the Cholesky factorisation and its implication for distributions on the space of correlation matrices, as presented in the previous two sections, can readily be extended to the complex and quaternion cases. To begin, augment the notation for the elements in (2.1) by writing $l_{jk}=l_{jk}^{(F)}$, where $(F)=r,c,q$ for the case of real, complex, quaternion entries respectively. In the complex case, one possible choice, which in fact occurs in the parametrisation of unitary matrices using Euler angles (see e.g. [4]), is

[TABLE]

where $0<\psi_{jk}<2\pi$. However in this parametrisation the Jacobian matrix for the change of variables from $\{\mathrm{Re}(l_{jk}^{(c)}),\mathrm{Im}(l_{jk}^{(c)})\}$ to $\{\psi_{jk},\theta_{jk}\}_{k=1,\cdots,j}$ is not triangular, making the calculations more difficult than need be.

Instead, we write $l_{jk}^{(c)}=l_{jk}^{(c),r}+il_{jk}^{(c),i}$ with $\mathrm{Re}(l_{jk}^{(c)})=l_{jk}^{(c),r}$, $\mathrm{Im}(l_{jk}^{(c)})=l_{jk}^{(c),i}$. Keeping in mind that $l_{jj}^{(c)}$ is required to be real, the analogue of (2.3) reads

[TABLE]

This suggests we view $(l_{j1}^{(c),r},l_{j1}^{(c),i},\cdots,l_{j,j-1}^{(c),r},l_{j,j-1}^{(c),i},l_{j,j}^{(c),r})$ as a point on the real sphere $S_{2j-1}$ and so introduce the parametrisation

[TABLE]

In the quaternion case the $j$-th row of $L$ in (1.5) consists of $2\times 2$ blocks of the form (4.13), each block to be denoted $l_{jk}^{(q)}$, ($k=1,\cdots,j$). The block $l_{jj}^{(q)}$ must represent a real number and this it is required $w=0$ and $z$ be real. For this block we set $z=l_{jj}^{(q),1}$. For the other blocks there are four real numbers corresponding to the real and imaginary parts of $z$ and $w$, which we denote $l_{jk}^{(q),s}$, ($s=1,\cdots,4$). In this setting the analogue of (2.3) reads

[TABLE]

suggesting that we view

[TABLE]

as a point on the real sphere $S_{4j-3}$. The corresponding hyperspherical parametrisation is

[TABLE]

with the (usual) convention that the products equal unity if they are empty.

Being effectively hyperspherical parametrisations of real spheres, we can write down the corresponding transformation in the volume forms associated with row $j$ in the coordinates (4.14) and (4). Thus these will involve the usual Jacobian in such a setting (see e.g. [13, Th.2.1.3]). In the complex case

[TABLE]

while in the quaternion case

[TABLE]

On the LHS of both (4.17) and (4.18) it is implicit that the volume form is restricted to the surface of the hypersphere.

We can make use of (4.14)–(4.18) to deduce the analogue of (3.1) in the complex and quaternion cases.

Proposition 7**.**

In the complex case

[TABLE]

while in the quaternion case

[TABLE]

Proof.

Without imposing the constraint that each diagonal entry in $R$ in (1.5) equals unity, but still requiring the diagonal entries of $L$ therein to be positive, the change of variables for the volume forms is specified by [5]

[TABLE]

where $\beta=1,2,4$ for $(F)=r,c,q$ and $l_{jj}^{(r),1}=l_{jj}^{(r)}$, $l_{jj}^{(c),1}=l_{jj}^{(c),r}$.

With $\rho_{jj}$ denoting the real diagonal entries of $R$ in (1.5), we have that

[TABLE]

These distributions substituted in (4.21), upon integrating over $\{\rho_{jj}\}$ on the LHS of (4.21), and over $\{l_{jj}^{(F),1}\}$ on the RHS impose the constraints that $\rho_{jj}=1$, $(j=1,\cdots,N)$.

Performing the integrations gives

[TABLE]

where on the RHS $\ast$ refers to the requirement that

[TABLE]

This latter constraint is built into the hyperspherical parametrisation. Making use then of the $j=k$ case in (4.14) and (4), and changing variables in $(\mathrm{d}L)|_{\{\ast\}}$ by forming the product over $j=2,\cdots,N$ of (4.17) (complex case) and of (4.18) (quaternion case), we read off from the resulting forms of the RHS of (4.23) the stated Jacobians. ∎

Suppose we now impose on the space of correlation matrices with complex or quaternion entries the probability distribution with density function

[TABLE]

in keeping with (3.5) in the real case. As is conventional in random matrix theory, in the quaternion case $\det R$ is defined as the square root of its value with $R$ represented in terms of the complex blocks (4.13). Hence, for $(F)=r,c,q$, we have $\det R=|\det L|^{2}=\prod_{l=1}^{N}\left(l_{jj}^{(F)}\right)^{2}.$ In terms of the hyperspherical parametrisation, reading from (2.4),(4.14) and (4) we thus have

[TABLE]

where the meaning of $\beta$ is as in (4.21). Combining (4.26) with (4.17) and (4.18) allows the normalisation $C^{(F)}$ to be evaluated, as for the derivation of (3.7).

Proposition 8**.**

With $\beta$ as in (4.21), the normalisation (4.25) has the explicit form

[TABLE]

Remark 9*.*

(1) Let $\begin{bmatrix}r_{jk}^{(F)}\end{bmatrix}_{j,k=1}^{N}$ be a Hermitian matrix with real, complex or quaternion entries for $F=r,c,q$ respectively. Suppose furthermore that all diagonal entries are equal to unity. As emphasized in [6] in the real case, $1/C^{F}_{0,N}$ is equal to the probability that when the real and imaginary parts (the latter for $F\neq r$) of the strictly upper triangular entries are chosen uniformly at random from $(-1,1)$, the matrix is positive definite and thus a correlation matrix.

(2) It has been commented in the paragraph containing (3.7) that the probability distribution (4.25) in the real case can be realised for $a=(n-N-1)/2$ by choosing $Y$ in (1.4) to be a standard Gaussian matrix of size $n\times N$. In [7, Exercises 3.3 q.3] this realisation is extended to all three cases with $a=(\beta/2)(n-N+1-2/\beta)$, $\beta$ as in (4.21), and $Y$ a $n\times N$ standard Gaussian matrix with entries from $F$.

Combining (4.26) with (4.17) and (4.18) also allows the complex and quaternion analogues of (3.9) and (3.10) to be obtained.

Proposition 10**.**

Let $\beta=1,2,4$ for $F=r,c,q$ respectively, and consider the situation that the random correlation matrix with elements from $F$ is chosen according to the probability distribution with density (4.25). We have that the marginal distribution of the real part of any single non-diagonal element of $R$ has probability density function

[TABLE]

We also have

[TABLE]

In the real case it was shown that the hyperspherical parametrisation (2.4) of the Cholesky factorisation (1.5) implies the simple parametrisation (2.9) of the partial correlations $\{\rho_{jk|\{1,\dots,k-1\}}\}_{1\leq k<j\leq N}$. Partial correlations are also well defined in the complex and quaternion cases through the formula (2.7). Note that in the quaternion case this quantity is a $2\times 2$ matrix with structure (4.13). Defining the complex and quaternion analogues of (2.10) and (2.12), we see that the working leading to (2.25) again holds true, with $A_{jk}^{(F)}=r_{j}l_{jk}^{(F)}$. We can thus write down from (4.14) and (4) the corresponding analogues of (2.9).

Proposition 11**.**

In the setting specified above, and with $1\leq k<j\leq N$ we have

[TABLE]

and

[TABLE]

with

[TABLE]

Acknowledgements

This work is part of a research program supported by the Australian Research Council (ARC) through the ARC Centre of Excellence for Mathematical and Statistical frontiers (ACEMS). PJF also acknowledges partial support from ARC grant DP170102028, and JZ acknowledges the support of a Melbourne postgraduate award, and an ACEMS top up scholarship.