On the Expected Value of the Determinant of Random Sum of Rank-One Matrices

TL;DR

This paper derives a simple formula for the expected determinant of a sum of random rank-one matrices, enabling efficient computation in applications like D-optimal estimation and random graphs.

Contribution

It introduces a novel, efficient method to compute the expected determinant of sums of random rank-one matrices, simplifying complex calculations.

Findings

Expected determinant can be computed via a single determinant calculation.

Applicable to D-optimal estimation and random graph analysis.

Reduces computational complexity from exponential to polynomial.

Abstract

We present a simple, yet useful result about the expected value of the determinant of random sum of rank-one matrices. Computing such expectations in general may involve a sum over exponentially many terms. Nevertheless, we show that an interesting and useful class of such expectations that arise in, e.g., D-optimal estimation and random graphs can be computed efficiently via computing a single determinant.

Equations

$$\mathbf{U}\triangleq\big{[}\mathbf{u}_{1}\,\,\mathbf{u}_{2}\,\,\cdots\,\,\mathbf{u}_{m}\big{]}\qquad\mathbf{V}\triangleq\big{[}\mathbf{v}_{1}\,\,\mathbf{v}_{2}\,\,\cdots\,\,\mathbf{v}_{m}\big{]}$$

$${\pi }_i$$

$$e(\mathbf{U},\mathbf{V},\mathbf{p})$$

$$\displaystyle=\mathbb{E}_{\boldsymbol{\pi}}\,\Big{[}\det\Big{(}\mathbf{U}\Pi\mathbf{V}^{\top}\Big{)}\Big{]}$$

$$e(\mathbf{U},\mathbf{V},\mathbf{p})$$

$$\displaystyle=\det\Big{(}\mathbf{U}\mathbf{P}\mathbf{V}^{\top}\Big{)},$$

$$\displaystyle\mathbb{E}_{\boldsymbol{\pi}}\,\Big{[}\det\Big{(}\sum_{i=1}^{m}\pi_{i}\mathbf{u}_{i}\mathbf{v}_{i}^{\top}\Big{)}\Big{]}$$

$$\displaystyle=\sum_{\mathcal{Q}\in\binom{[m]}{n}}\mathbb{E}_{\boldsymbol{\pi}}\,\Big{[}\det\Big{(}\sum_{k\in\mathcal{Q}}\pi_{k}\mathbf{u}_{k}\mathbf{v}_{k}^{\top}\Big{)}\Big{]}.$$

$$\displaystyle\det\Big{(}\sum_{k\in\mathcal{Q}}\pi_{k}\mathbf{u}_{k}\mathbf{v}_{k}^{\top}\Big{)}=\begin{cases}\det\Big{(}\sum_{k\in\mathcal{Q}}\mathbf{u}_{k}\mathbf{v}_{k}^{\top}\Big{)}&\text{iff $\pi_{k}=1$ for all $k\in\mathcal{Q}$,}\\ 0&\text{otherwise.}\end{cases}$$

$$\mathbb{P}\,\Big{[}\bigwedge_{k\in\mathcal{Q}}\pi_{k}=1\Big{]}=\prod_{k\in\mathcal{Q}}p_{k}.$$

$$\displaystyle\mathbb{E}_{\boldsymbol{\pi}}\,\Big{[}\det\Big{(}\sum_{k\in\mathcal{Q}}\pi_{k}\mathbf{u}_{k}\mathbf{v}_{k}^{\top}\Big{)}\Big{]}$$

$$\displaystyle=\det\Big{(}\sum_{k\in\mathcal{Q}}p_{k}\mathbf{u}_{k}\mathbf{v}_{k}^{\top}\Big{)}.$$

$$\displaystyle\mathbb{E}_{\boldsymbol{\pi}}\,\Big{[}\det\Big{(}\sum_{i=1}^{m}\pi_{i}\mathbf{u}_{i}\mathbf{v}_{i}^{\top}\Big{)}\Big{]}$$

$$\mathbf{z}=\mathbf{Hx}+ϵ\text{where}ϵ\sim \mathcal{N}(0,\Sigma )$$

$$\hat{\mathbf{x}}={({\bar{\mathbf{H}}}^{\top }\bar{\mathbf{H}})}^{-1}{\bar{\mathbf{H}}}^{\top }\mathbf{z}.$$

$$\operatorname{Cov}\,[\hat{\mathbf{x}}]={\big{(}\bar{\mathbf{H}}^{\top}\bar{\mathbf{H}}\big{)}}^{-1}.$$

$$t_w(\mathcal{G})$$

$$=\det ({\mathbf{L}}_{\mathcal{G}})$$

$$\displaystyle\mathbb{E}_{\pi}\,\Big{[}t_{w}(\mathcal{G}_{\pi})\Big{]}=\mathbb{E}_{\pi}\,\Big{[}\det\Big{(}\sum_{i=1}^{m}\pi_{i}w(e_{i})\,\mathbf{a}_{i}\mathbf{a}_{i}^{\top}\Big{)}\Big{]}.$$

$$\displaystyle\mathbb{E}_{\pi}\,\Big{[}t_{w}(\mathcal{G}_{\pi})\Big{]}$$

$$=e({\mathbf{A}}_w,{\mathbf{A}}_w^{\top },\mathbf{p})$$

$$\displaystyle=\det\Big{(}\sum_{i=1}^{m}p_{i}w(e_{i})\,\mathbf{a}_{i}\mathbf{a}_{i}^{\top}\Big{)}$$

$$=\sum _{\mathcal{T}\in \mathbb{T}(\mathcal{G})}\prod _{e_i\in \mathcal{E}(\mathcal{T})}{p}_{i}w(e_i).$$

$$\displaystyle\mathbb{E}_{\pi}\,\Big{[}t_{w}(\mathcal{G}_{\pi})\Big{]}$$

$$\displaystyle\mathbb{E}_{\boldsymbol{\pi}}\,\Big{[}\det\Big{(}\sum_{i=1}^{m}\pi_{i}\mathbf{U}_{i}\mathbf{V}_{i}^{\top}\Big{)}\Big{]}$$

$$\displaystyle\mathbb{E}_{\pi}\,\Big{[}t_{w}(\mathcal{G}_{\pi})\Big{]}$$

$$=\sum _{\mathcal{T}\in \mathbb{T}(\mathcal{G})}\prod _{e_i\in \mathcal{E}(\mathcal{T})}{p}_{b_i}^{1/n_{b_i}(\mathcal{T})}w(e_i)$$

Full text

1 Problem Definition

• •

$[n]\triangleq\{1,2,\dots,n\}$, and for any finite set $\mathcal{W}$, $\binom{\mathcal{W}}{k}$ is the set of $k$-subsets of $\mathcal{W}$.

• •

Suppose we are given a pair of $m$ real $n$-vectors, $\{\mathbf{u}_{i}\}_{i=1}^{m}$ and $\{\mathbf{v}_{i}\}_{i=1}^{m}$. Define,

[TABLE]

• •

Let $\{\pi_{i}\}_{i=1}^{m}$ be $m$ independent Bernoulli random variables distributed as,

[TABLE]

where $\{p_{i}\}_{i=1}^{m}$ are given. Define $\mathbf{p}\triangleq[p_{1}\,\,p_{2}\,\,\cdots\,\,p_{m}]^{\top}$ and $\boldsymbol{\pi}\triangleq[\pi_{1}\,\pi_{2}\,\cdots\,\pi_{m}]^{\top}.$

• •

We are interested in computing the expression below,

[TABLE]

where $\Pi\triangleq\operatorname{diag}(\pi_{1},\pi_{2},\dots,\pi_{m})$. Note that the naive way of computing this expectation leads to a computationally intractable sum over $\{0,1\}^{m}$.

2 Main Result

Theorem 1** (Main Result [4]).**

[TABLE]

where $\mathbf{P}\triangleq\operatorname{diag}(p_{1},p_{2},\dots,p_{m})$.

Proof of Theorem 1.

The proof outline is as follows:

1. Step 1.

First, the Cauchy-Binet formula is used to expand the determinant as a sum over $\binom{m}{n}$ terms. 2. Step 2.

The expected value of each of the $\binom{m}{n}$ terms can be easily computed. 3. Step 3.

Finally, the Cauchy-Binet formula is applied again to shrink the sum.

We begin by applying the Cauchy-Binet formula:

[TABLE]

Since $|\mathcal{Q}|=n$, we will have \operatorname{rank}\Big{(}\sum_{k\in\mathcal{Q}}\pi_{k}\mathbf{u}_{k}\mathbf{v}_{k}^{\top}\Big{)}<n if there exists $k\in\mathcal{Q}$ for which $\pi_{k}=0$. Hence, the determinant can be non-zero only when $\pi_{k}=1$ for all $k\in\mathcal{Q}$. Therefore,

[TABLE]

But from the independence assumption we know that,

[TABLE]

Each individual expectation in (9) can be computed as follows.

[TABLE]

Plugging (13) back into (9) yields,

[TABLE]

Note that (14) is nothing but the Cauchy-Binet expansion of \det\Big{(}\sum_{i=1}^{m}p_{i}\mathbf{u}_{i}\mathbf{v}_{i}^{\top}\Big{)}. This concludes the proof. ∎

3 Motivation & Applications

$e(\mathbf{U},\mathbf{V},\mathbf{p})$ arises in the following problems:

Estimation

Suppose $\mathbf{x}\in\mathbb{R}^{n}$ is an unknown quantity to be estimated using $m$ observations $\{z_{i}\}_{i=1}^{m}$ ($m\geq n$) generated according to,

[TABLE]

where $\mathbf{z}\triangleq[z_{1}\,\,z_{2}\,\,\cdots\,\,z_{m}]^{\top}$. To simplify our notation, let us define $\bar{\mathbf{H}}\triangleq\Sigma^{-1/2}\mathbf{H}$. The maximum likelihood estimator $\hat{\mathbf{x}}$ has the following form:

[TABLE]

It is well known that $\hat{\mathbf{x}}$ is unbiased and efficient; i.e., it achieves the Cramé-Rao lower bound,

[TABLE]

Geometrically speaking, the hypervolume of uncertainty hyperellipsoids are proportional to $\sqrt{\det\operatorname{Cov}\,[\hat{\mathbf{x}}]}$ (see, e.g., [2]). The D-optimality (determinant-optimality) criterion is defined as $\det\operatorname{Cov}\,[\hat{\mathbf{x}}]^{-1}$. Note that $\det\operatorname{Cov}\,[\hat{\mathbf{x}}]={(\det\,\mathcal{F})}^{-1}$ where $\mathcal{F}\triangleq\bar{\mathbf{H}}^{\top}\bar{\mathbf{H}}$ is the so-called Fisher information matrix. Hence, minimizing the determinant of the estimation error covariance matrix is equivalent to maximizing the D-optimality criterion, $\det\,(\bar{\mathbf{H}}^{\top}\bar{\mathbf{H}})$. Now consider the following scenarios.

1. (a)

Sensor Failure: The $i$th “sensor” may “fail” independently with probability $1-p_{i}$, for all $i\in[m]$. In this case, the row corresponding to each failed sensor has to be removed from $\bar{\mathbf{H}}$. Hence, $e(\bar{\mathbf{H}}^{\top},\bar{\mathbf{H}},\mathbf{p})$ gives the expected value of the D-optimality criterion. 2. (b)

Sensor Selection: The goal in D-optimal sensor selection is to select a subset (e.g., $k$-subset) of $m$ available sensors (observations) such that the D-optimality criterion is maximized. Joshi and Boyd [2] proposed an approximate solution to this problem through convex relaxation. In [4], we showed that their convex program can be interpreted as the problem of finding the optimal probabilities $\{p_{i}\}_{i=1}^{m}$ for randomly selecting (e.g., $k$) sensors via independent coin tosses such that the expected value of the D-optimality criterion, i.e., $e(\bar{\mathbf{H}}^{\top},\bar{\mathbf{H}},\mathbf{p})$, is maximized. See [4, 3] for the details.

Remark 1**.**

*For sufficiently smooth nonlinear measurement models, $\bar{\mathbf{H}}$ should be replaced by the normalized Jacobian of the measurement function. *

**Spanning Trees in Random Graphs111We first presented Theorem 1, and its special case used for computing the weighted number of spanning trees, in [4]. Recently we discovered an earlier result for computing the expected number of spanning trees in unweighted anisotropic random graphs by Joel E. Cohen in 1986 [1]. Cohen in [1] provides a different proof and extends his result to the case of random directed graphs. Our result, however, considers the weighted graphs, while our Theorem 1 extends it to the general case of random sum of arbitrary rank-one matrices. **

Networks with “reliable” (against, e.g., noise in estimation, or failure in communication) topologies are crucial in many applications across science and engineering. In general, the notion of reliability in networks is closely related to graph connectivity. Among the existing combinatorial and spectral graph connectivity criteria, the number of spanning trees (sometimes referred to as graph complexity or tree-connectivity) stands out: despite its combinatorial origin, it can also be characterized solely by the spectrum of the graph Laplacian (Kirchhoff) matrix. This result is due to Kirchhoff’s matrix-tree theorem (and its extensions):

Theorem 2** (Kirchhoff’s Matrix-Tree Theorem for Weighted Graphs).**

Consider graph $\mathcal{G}=(\mathcal{V},\mathcal{E},w)$ where $\mathcal{V}=\{v_{i}\}_{i=0}^{n}$, $\mathcal{E}\subseteq\binom{\mathcal{V}}{2}$, and $w:\mathcal{E}\to\mathbb{R}_{>0}$. The reduced Laplacian matrix of $\mathcal{G}$, denoted by $\mathbf{L}_{\mathcal{G}}$, is obtained by removing an arbitrary row and the corresponding column from the (weighted) Laplacian matrix of $\mathcal{G}$; e.g., $v_{0}$. The weighted number of spanning is given by,

[TABLE]

where $\mathbb{T}(\mathcal{G})$ is the set of all spanning trees of $\mathcal{G}$, and $\mathcal{E}(\mathcal{T})$ denotes the edge set of graph $\mathcal{T}$. Note that in case of unit weights, $t_{w}(\mathcal{G})$ is simply the number of spanning trees in $\mathcal{G}$.

Now consider a random graph whose $i$th edge is “operational” with probability $p_{i}$, independent of other edges (Figure 1).222Here, “operational” means that the corresponding vertices are connected via an edge. Define indicator variables $\{\pi_{i}\}_{i=1}^{m}$ such that $\pi_{i}=1$ iff the $i$th edge is operational, otherwise $\pi_{i}=0$. The reduced (unweighted) incidence matrix of $\mathcal{G}$, $\mathbf{A}=[\mathbf{a}_{1}\,\mathbf{a}_{2}\,\cdots\,\mathbf{a}_{m}]$, is obtained by removing an arbitrary row from the (unweighted) incidence matrix of $\mathcal{G}$. From Theorem 2 we know that,

[TABLE]

Define $\mathbf{A}_{w}\triangleq\mathbf{A}\sqrt{\mathbf{W}}$ in which \mathbf{W}\triangleq\operatorname{diag}\big{(}w(e_{1})\,w(e_{2})\,\cdots\,w(e_{m})\big{)}. Note that this expression is equal to $e(\mathbf{A}_{w},\mathbf{A}_{w}^{\top},\mathbf{p})$. From Theorem 1 we have,

[TABLE]

Remark 2**.**

It is worth mentioning that, according to above equations, the expected weighted number of spanning trees is given by computing the weighted number of spanning trees after multiplying the edge weights by their probabilities; i.e.,

[TABLE]

where $w_{p}:e_{i}\mapsto p_{i}w(e_{i})$.

4 Random Sum of Rank-$r$ Matrices

It is not immediately clear whether there is an efficient way for computing

[TABLE]

in which $\mathbf{U}_{i}$ and $\mathbf{V}_{i}$ belong to $\mathbb{R}^{n\times r_{i}}$ for $i\in[m]$. Nevertheless, the following results provide some preliminary insights into this more general case. The proofs of the following lemmas follow that of Theorem 1—i.e., Cauchy-Binet formula.

Lemma 1**.**

[TABLE]

Lemma 2**.**

Consider a random graph $\mathcal{G}_{\pi}$ (over graph $\mathcal{G}$) whose edge set $\mathcal{E}$ is partitioned into $k$ blocks $\{\mathcal{E}_{i}\}_{i=1}^{k}$. The edges in the $i$th block are operational, independent of other blocks, with probability $p_{i}$. Let $\mathbf{A}_{i}$ be the collection of the columns of the reduced weighted incidence matrix that belong to the $i$th block of edges $\mathcal{E}_{i}$. We have,

[TABLE]

where $b_{i}$ is the block index that contains $e_{i}$ and $n_{i}(\mathcal{T})\triangleq|\mathcal{E}(\mathcal{T})\cap\mathcal{E}_{i}|$.