Testing the Structure of a Gaussian Graphical Model with Reduced Transmissions in a Distributed Setting

TL;DR

This paper proposes a distributed testing method for Gaussian graphical models that reduces transmissions by ordering sensor communications, achieving near-optimal performance with fewer data exchanges.

Contribution

It introduces an ordered transmission scheme for distributed GGM testing that maintains optimal Bayes risk while significantly reducing communication overhead.

Findings

Achieves the same Bayes risk as centralized testing with fewer transmissions.

Transmission savings approach half the number of clusters under certain conditions.

Guarantees a lower bound on transmission reduction for any GGM.

Abstract

Testing a covariance matrix following a Gaussian graphical model (GGM) is considered in this paper based on observations made at a set of distributed sensors grouped into clusters. Ordered transmissions are proposed to achieve the same Bayes risk as the optimum centralized energy unconstrained approach but with fewer transmissions and a completely distributed approach. In this approach, we represent the Bayes optimum test statistic as a sum of local test statistics which can be calculated by only utilizing the observations available at one cluster. We select one sensor to be the cluster head (CH) to collect and summarize the observed data in each cluster and intercluster communications are assumed to be inexpensive. The CHs with more informative observations transmit their data to the fusion center (FC) first. By halting before all transmissions have taken place, transmissions can be saved without performance loss. It is shown that this ordering approach can guarantee a lower bound on the average number of transmissions saved for any given GGM and the lower bound can approach approximately half the number of clusters when the minimum eigenvalue of the covariance matrix under the alternative hypothesis in each cluster becomes sufficiently large.

Equations

$${\bigg{[}({\bf A}_{\cal U})^{-1}\bigg{]}^{\cal V}_{i,j}}=\left\{{\begin{array}[]{*{20}{l}}{{{\left({\bf{A}}^{-1}\right)}_{i,j}}},&{\text{ if }i\in{\cal U},\;j\in{\cal U}}\\ 0,&\text{ otherwise. }\end{array}}\right.$$

$${\left({\mathbf{\Sigma }}^{-1}\right)}_{i,j}=0$$

$${\mathcal{H}}_k={\mathcal{C}}_1\cup {\mathcal{C}}_2\cup \cdots \cup {\mathcal{C}}_k,\forall k=1,2,...,K,$$

$${\mathcal{S}}_k={\mathcal{H}}_{k-1}\cap {\mathcal{C}}_k,\forall k=2,...,K.$$

$$\frac{q\left(k\right)\text{buildrel}\Delta }{=\min \left\{j|{\mathcal{S}}_k\subseteq {\mathcal{C}}_j\right\},\forall k=2,3,...,K.}$$

$${\mathcal{S}}_k\subseteq {\mathcal{C}}_{q(k)}.$$

$${\mathcal{S}}_k\subseteq {\mathcal{C}}_k.$$

$$q(k)<k$$

$$\frac{{\mathcal{Q}}_j\text{buildrel}\Delta }{=\left\{k|q\left(k\right)=j\right\}.}$$

$$\min {\mathcal{Q}}_j>j,$$

$${\mathcal{Q}}_j\subseteq \left\{j+1,j+2,...,K\right\},\forall j=1,2,...,K-1,$$

$${\mathcal{Q}}_K=\varnothing .$$

$${\mathbf{\Sigma }}^{-1}=\sum _{k=1}^K{[({\mathbf{\Sigma }}_{{\mathcal{C}}_k}{)}^{-1}]}^{\mathcal{V}}-\sum _{k=2}^K{[({\mathbf{\Sigma }}_{{\mathcal{S}}_k}{)}^{-1}]}^{\mathcal{V}}.$$

$$\displaystyle\det{\bf\Sigma}=\Big{(}\prod\limits_{k=1}^{K}\det{\bf\Sigma}_{{\cal C}_{k}}\Big{)}\bigg{/}\Big{(}{{\prod\limits_{k=2}^{K}\det{\bf\Sigma}_{{\cal S}_{k}}}}\Big{)}.$$

$$H_0:\text{x}\sim \mathcal{N}(0,\mathbf{I})$$

$$H_1:\text{x}\sim \mathcal{N}(0,\mathbf{\Sigma })$$

$${\delta_{B}}({\bf{x}})=\left\{\begin{array}[]{l}1\;\text{ if }T({\bf x})\geq 2\tau\\ 0\;\text{ if }T({\bf x})<2\tau\end{array}\right.,$$

$$\frac{\tau \text{buildrel}\Delta }{=\ln {\pi }_0/{\pi }_1}$$

$$T(\mathbf{x})$$

$$=2\ln \frac{{(2\pi )}^{-\frac{N}{2}}{(\det \mathbf{\Sigma })}^{-\frac{1}{2}}\exp \{-\frac{1}{2}{\mathbf{x}}^{\mathbf{T}}{(\mathbf{\Sigma })}^{-1}\mathbf{x}\}}{{(2\pi )}^{-\frac{N}{2}}{(\det \mathbf{I})}^{-\frac{1}{2}}\exp \{-\frac{1}{2}{\mathbf{x}}^{\mathbf{T}}{(\mathbf{I})}^{-1}\mathbf{x}\}}$$

$$={\mathbf{x}}^{\mathbf{T}}{\mathbf{I}}^{-1}\mathbf{x}-{\mathbf{x}}^{\mathbf{T}}{\mathbf{\Sigma }}^{-1}\mathbf{x}$$

$$-\ln \det (\mathbf{\Sigma })+\ln \det (\mathbf{I})>2\tau .$$

$$\displaystyle T({\bf x})=\sum_{k=1}^{K}\big{(}{{\bf{x}}^{T}_{{{\cal C}_{K}}}}{{{{{\bf{J}}_{k}}}}}{{\bf{x}}_{{{\cal C}_{K}}}}-{e}_{k}\big{)}=\sum\limits_{k=1}^{K}{{L_{k}}({{\bf{x}}_{{{\cal C}_{k}}}})}$$

$${\mathbf{J}}_1$$

$${\mathbf{J}}_k$$

$$\displaystyle\qquad-\alpha_{k}\Big{[}{{\bf I}_{{{\cal S}_{k}}}^{-1}}-({\bf{\Sigma}}_{{{\cal S}_{k}}})^{-1}\Big{]}^{{{\cal C}_{k}}}$$

$${\alpha }_k+{\beta }_k=1,\forall k=2,3,...,K.$$

$$e_1=\ln \det ({\mathbf{\Sigma }}_{{\mathcal{C}}_1})-\sum _{j\in {\mathcal{Q}}_1}{\beta }_j\ln \det ({\mathbf{\Sigma }}_{{\mathcal{S}}_j}),$$

$$e_k$$

$$|{\tilde{L}}_1|>|{\tilde{L}}_2|>\cdots |{\tilde{L}}_K|.$$

$$\frac{{\tau }_U\text{buildrel}\Delta }{=2\tau +n_{UT}|{\tilde{L}}_{K-n_{UT}}|}$$

$$\frac{{\tau }_L\text{buildrel}\Delta }{=2\tau -n_{UT}|{\tilde{L}}_{K-n_{UT}}|,}$$

$$P_{D,k}(\delta )=\mathrm{Pr}\left(L_k\left({\mathbf{x}}_{{\mathcal{C}}_k}\right)>\delta |H_1\right)$$

Full text

I Introduction

Great attention has been devoted to distributed detection in sensor networks for both military and civilian applications, such as security, tactical surveillance, defense operations, disaster prediction and health care monitoring [1]. However, since spatially distributed sensor nodes have limited battery capacity, energy efficiency is an important topic [2][3][4][5]. Recently, for the shift-in-mean detection problem, [4] proposed an approach where transmissions can be ordered and halted before all transmissions have taken place based on the assumption that the observations are independent and identically distributed. The work in [5] has generalized the ordered transmission approaches to mean-shift detection with statistically dependent observations. However, covariance matrix testing problems employing ordered transmissions have not been addressed yet. This paper differs from the previous work on ordered transmissions for the shift-in-mean problem [4][5], and focuses on problems testing the covariance matrix [6][7] by employing a Gaussian Graphical Model (GGM) formulation.

A GGM characterizes dependencies using a graph with a node for each variable and edges labeled with weights that describe the correlation between the connected nodes. The observations of each node are assumed to follow a Gaussian distribution and the dependency between the observations is represented by the edge. Since GGMs can be used to describe high dimensional parametric models with a small number of samples via message passing algorithms, they have received extensive study in speech recognition [8], Internet backbone networks [9], genetic networks [10], image processing [11], machine learning [12][13], sensor networks [14] [15] and electrical power systems [6][16][17]. In this paper, we employ the GGM to organize the sensors into clusters based on the largest fully connected subgraphs they belong to, and this provides the possibility of applying ordered transmissions.

We consider testing the graphical structure of a decomposable GGM with reduced transmissions in a distributed scenario. We first group all sensors into several clusters. These clusters correspond to the largest fully connected subgraphs (called cliques). Then we show we can decompose the global optimum test statistic into a sum of local test statistics, each of which only depends on the observations in the corresponding cluster. Each cluster selects one sensor as the cluster head (CH) which we assume will have more computation and communication capacity [18]. The other sensors in the cluster transmit their observations to the CH and then the CH calculates the local test statistic and transmits it to the fusion center (FC). Typically the sensors in a cluster (clique) are located close to each other111The correlation structure implied by a clique will usually imply close physical proximity in the applications we have surveyed, see [16] for example., so local communications within the cluster do not use as much energy as the communications from the CH to the FC. We employ ordered transmissions over the CHs to reduce the more costly communications from the CHs to the FC. Specifically, after collecting and summarizing the observations from other sensors in the cluster, the CH sets a timer to decide when to transmit its local test statistic. All CHs are assumed to be synchronized and the timer in each cluster is inversely proportional to the magnitude of the local test statistic so that the CH with the most informative observations will transmit first, followed by the next most informative CH, and so on. This provides a method to apply ordered transmissions to solve covariance matrix testing problems with dependent observations. Developing an ordering algorithm for covariance matrix testing problems is one contribution of this work. The second major contribution is that a lower bound on the average number of transmissions saved is derived in this paper which is a valid lower bound for all cases we consider. The last contribution is that we have shown that when the minimum eigenvalue of the covariance matrix under the alternative hypothesis in each cluster becomes sufficiently large, nearly half of the transmissions can be omitted which provides a limiting behavior of the general lower bound on the number of transmissions saved.

I-A Notation and Organization

Throughout this paper, bold lower case letters are used to denote column vectors, and bold upper case letters denote matrices. For any vector ${\bf x}\in{\mathbbm{R}}^{N}$ and any set ${\cal S}\subseteq\{1,2,...,N\}$, the vector ${\bf x}_{\cal S}$ is defined as a subvector of $\bf x$ which consists of the elements of $\bf x$ corresponding to the indices contained in $\cal S$. For any matrix ${\bf A}$, let ${({\bf{A}})_{i,j}}$ denote the element in the $i$-th row and $j$-th column. ${\bf{A}}\succ 0$ and ${\bf{A}}\succeq 0$ imply that the matrix $\bf A$ is positive definite and positive semidefinite, respectively. The eigenvalues of $\bf A$ are denoted as $eig\{{\bf A}\}$. Given any two sets of indices ${\cal U}$ and ${\cal V}$ where ${\cal U}\subseteq{\cal V}$, let $\bf A$ denote a matrix whose coordinates correspond to the indices in ${\cal U}$, then $[({\bf A}_{\cal U})^{-1}]^{\cal V}$ denotes the matrix obtained from the appropriate zero-filling needed to obtain a dimension $|{\cal V}|$-by-$|{\cal V}|$ matrix as per

[TABLE]

For example, if we let ${\bf{A}}^{-1}=[1,1;1,1]$, ${\cal U}=\{2,3\}$ and ${\cal V}=\{1,2,3\}$, then we obtain $[({\bf A}_{\cal U})^{-1}]^{\cal V}=[0,0,0;0,1,1;0,1,1]$.

The rest of paper is organized as follows. The basics of decomposable GGMs, the covariance testing problem and the distributed test statistic are presented in Section II. Ordering for covariance matrix testing problems and a lower bound on the average number of transmissions saved via ordering are provided in Section III. Performance analysis and numerical examples are presented in Section IV. Finally, conclusions are drawn in Section V.

II Covariance Matrix Testing in Decomposable Gaussian Graphical Models

In this section, we introduce the properties of decomposable GGMs and formulate the multivariate Gaussian covariance matrix testing problem in a distributed scenario.

II-A Decomposable Gaussian Graphical Models

Consider an undirected graph ${\cal G}=\left({{\cal V},{\cal E}}\right)$ where ${\cal V}=\{1,2,...,N\}$ is the set of indices and ${\cal E}=\{(i_{1},j_{1}),(i_{2},j_{2}),...,(i_{|{\cal E}|},j_{|{\cal E}|})\}$ denotes the set of undirected edges of the graph. Let a random vector ${\bf{x}}\buildrel\Delta\over{=}{\left[{{x_{1}},{x_{2}},...,{x_{N}}}\right]^{T}}$ follow a multivariate Gaussian distribution which also satisfies the Markov property with respect to ${\cal G}$ which implies that if $(i,j)\notin{\cal E}$, then

[TABLE]

where ${\bf{\Sigma}}$ is the covariance matrix of $\bf x$, and ${{{\bf{\Sigma}}^{-1}}}$ is referred to as the concentration matrix (also known as the information matrix). Note that the topology of the graph $\cal G$ is described by the non-zero elements of the concentration matrix ${{{\bf{\Sigma}}^{-1}}}$.

An undirected graph is decomposable if it can be successively decomposed into its cliques [19]. Throughout the paper, we concentrate on decomposable undirected graph models. Let $K$ denote the number of cliques in the decomposable undirected graph ${\cal G}$. The perfect sequence of cliques of the graph ${\cal G}$ is denoted by $\{{\cal C}_{1},{\cal C}_{2},...,{\cal C}_{K}\}$. We denote the corresponding histories $\{{\cal H}_{k}\}_{k=1,2,...,K}$ and separators $\{{\cal S}_{k}\}_{k=2,3,...,K}$ as

[TABLE]

and

[TABLE]

Note that for all $k>1$, there is a $j<k$ such that ${\cal S}_{k}\subseteq{\cal C}_{j}$ for any decomposable undirected graph $\cal G$.

A mapping $q:\{{2,3,...,K}\}\to\{{1,2,....,K}\}$ is defined to specify an association between each separator set and some unique cliques such that [5]

[TABLE]

Thus, the $k$-th separator ${\cal S}_{k}$ is associated with the $q(k)$-th clique ${{\cal C}_{q(k)}}$ according to

[TABLE]

Note that the $k$-th separator ${\cal S}_{k}$ is not only contained in the $q(k)$-th clique ${\cal C}_{q(k)}$, but also contained in the $k$-th clique ${{\cal C}_{k}}$ as defined in (4), that is,

[TABLE]

It is worth mentioning that for any $k>1$, $q(k)$ must exist and

[TABLE]

for any decomposable undirected graph $\cal G$. Let ${\cal Q}_{j}$ denote the set of indices of the separators which are associated with the $j$-th clique via the mapping $q$ in (5), that is,

[TABLE]

From (8), we know that the minimal element in ${{\cal Q}_{j}}$ satisfies

[TABLE]

which implies that

[TABLE]

and

[TABLE]

Consider the example in Fig. 1, by employing (5), we observe that $q(2)=q(3)=q(4)=1$ and $q(5)=2$ which implies ${{\cal S}_{2}}\subseteq{{\cal C}_{1}}$, ${{\cal S}_{3}}\subseteq{{\cal C}_{1}}$, ${{\cal S}_{4}}\subseteq{{\cal C}_{1}}$ and ${{\cal S}_{5}}\subseteq{{\cal C}_{2}}$. By employing (7), we also obtain ${{\cal S}_{2}}\subseteq{{\cal C}_{2}}$, ${{\cal S}_{3}}\subseteq{{\cal C}_{3}}$, ${{\cal S}_{4}}\subseteq{{\cal C}_{4}}$ and ${{\cal S}_{5}}\subseteq{{\cal C}_{5}}$. Note that $q(2)<2$, $q(3)<3$, $q(4)<4$ and $q(5)<5$. By employing (9), we obtain ${{\cal Q}_{1}}=\{2,3,4\}$ and ${{\cal Q}_{2}}=\{5\}$ but ${{\cal Q}_{3}}={{\cal Q}_{4}}={{\cal Q}_{5}}=\emptyset$.

Let ${\bf x}_{{{{\cal C}_{k}}}}$ denote the set of observations in ${\bf x}$ that come from the nodes in the $k$-th clique. Let ${\bf x}_{{{{\cal S}_{k}}}}$ denote the observations in ${\bf x}$ that come from the nodes in the $k$-th separator set. Let ${{{\bf{\Sigma}}_{{{\cal C}_{k}}}}}$ and ${{{\bf{\Sigma}}_{{{\cal S}_{k}}}}}$ denote the covariance matrices associated with ${\bf x}_{{{{\cal C}_{k}}}}$ and ${\bf x}_{{{{\cal S}_{k}}}}$ respectively. The information matrix of x can be expressed by those of the cliques and separators in decomposable GGMs as [19]

[TABLE]

Fig. 2 gives a numerical example of (13), using a chain structure (see Fig. 3). Here, there are $K=2$ clusters, and each cluster has 3 nodes, resulting in 3 terms on the right-hand side of (13).

Also, for a decomposable GGM [19],

[TABLE]

II-B Gaussian Covariance Matrix Testing Problems

Consider a network with $N$ sensors and the Gaussian covariance matrix testing problem given by

[TABLE]

where the covariance matrix $\bf\Sigma$ is assumed to be positive definite and without loss of generality we assume the identity matrix under $H_{0}$, because (II-B) can be obtained via pre-whitening.

The Bayes optimum decision rule for (II-B) is a log-likelihood ratio (LLR) threshold test

[TABLE]

and the threshold $\tau$ is

[TABLE]

where ${\pi_{0}}$ and ${\pi_{1}}$ are the a priori probabilities of $H_{0}$ and $H_{1}$ respectively. The detector decides $H_{1}$ if

[TABLE]

The optimum centralized energy unconstrained detection approach requires each sensor to send its observation to a FC. After receiving the data from all sensors, the FC employs (II-B). In our distributed approach, we partition the sensors into $K$ clusters which correspond to the cliques. The CH will collect the information from the sensors in a given cluster and then transmit it to the FC. By employing ordering over the clusters, we reduce the number of CH to FC transmissions while achieving the same probability of error as the optimum centralized energy unconstrained detection approach. Throughout the paper, the following assumption is made.

Assumption 1

Every sensor is physically close to its neighboring sensors in the graph. Hence, sensors in the same cluster are physically close so that intercluster communications used by CHs are assumed to be short distances so we only focus on communications between the CHs and the FC.

Plugging (13) and (14) into (II-B), the test statistics $T(\bf x)$ can be rewritten as (refer to Appendix A for details)

[TABLE]

where

[TABLE]

and

[TABLE]

for $k=2,3,...,K$ where ${\cal Q}_{j}$ for $j=1,2,...,K$ is defined in (9) and the set of coefficient pairs $\{(\alpha_{k},\beta_{k})\}_{k=2}^{K}$ satisfies

[TABLE]

Note that there are uncountably many choices of $\{\alpha_{k},\beta_{k}\}_{k=2}^{K}$ satisfying (22) which implies that multiple $T({\bf x})$ in (19) will work where each splits the observations a little differently over the clusters via nonuniqueness of separator sets. In (19),

[TABLE]

and for all $k=2,3,...,K$,

[TABLE]

We refer to the $k$-th local test statistic (LTS) ${{L_{k}}({{\bf{x}}_{{{\cal C}_{k}}}})}$ as the local log-likelihood ratio (LLLR), since each ${L_{k}}\left({{{\bf{x}}_{{{\cal C}_{k}}}}}\right)$ can be calculated by only utilizing the observations available in the $k$-th cluster for $k=1,2,...,K$.

III Reduced Transmission Energy Detection Using Ordered Transmission Approach

III-A Ordering for the Covariance Testing Problem

The following theorem describes our distributed algorithm (ordering) to reduce transmissions [4].

Theorem 1

Consider an approach where the $k$-th CH will transmit its ${{L_{k}}({{\bf{x}}_{{{\cal C}_{k}}}})}$ from (19) after a time equal to $\eta/|{{L_{k}}({{{\bf{x}}_{{{\cal C}_{k}}}}})}|$, with $\eta$ any real positive number. All transmissions stop when the sum of the ${{L_{k}}({{\bf{x}}_{{{\cal C}_{k}}}})}$ received is larger than a threshold $\tau_{U}$ or smaller than a threshold $\tau_{L}$. The ordered magnitudes of $\{L_{k}({\bf x}_{{\cal C}_{k}})\}_{k=1}^{K}$ are denoted as

[TABLE]

Let $n_{UT}<K$ be the number of CHs who have not yet transmitted at a given time and let ${\tilde{L}}_{K-n_{UT}}$ denote the last LLLR transmission. Define

[TABLE]

and

[TABLE]

where $\tau$ is defined in (17). The approach described in this theorem gives the same probability of error as the optimum centralized energy unconstrained approach where all CHs transmit their LLLRs to the FC, while using a smaller average number of transmissions.

Proof:

The proof of this theorem is omitted, since it follows the proof in [4]. ∎

As demonstrated by Theorem 1, every time the FC receives a new CH transmission, it updates the thresholds and compares them with the sum of the CH transmissions it has received so far, denoted by $L^{\prime}$. If $L^{\prime}\geq{\tau}_{U}$ (or $L^{\prime}<{\tau}_{L}$), then the test statistic $T({\bf x})$ from (II-B) is guaranteed to be larger (or smaller) than the optimum threshold $2\tau$, regardless of the CH transmissions that have not yet been transmitted. Then the FC can send a message to the CHs to stop the remaining transmissions. By halting before all transmissions have taken place, transmissions can be saved without performance loss when sufficient evidence is accumulated. If ${\tau}_{L}\leq L^{\prime}<{\tau}_{U}$, which means the FC has not accumulated sufficient evidence yet, then transmissions continue. Note that the ordering approach is only applied to the global communications (communications from the CHs to the FC), but not local communications (intercluster communications).

III-B Lower Bound on the Average Number of Transmissions Saved

For any given $\delta$, we define the detection probability $P_{D,k}$ and the false alarm probability $P_{f,k}$ in the $k$-th cluster for $k=1,2,...,K$ as

[TABLE]

and

[TABLE]

We provide the following theorem with regard to a lower bound on the average number of transmissions saved by ordering the global communications.

Theorem 2

Consider the hypothesis testing problem described in (II-B). Let

[TABLE]

and

[TABLE]

where the threshold $\tau$ is defined in (17). When using the ordered transmission approach described in Theorem 1, if $K>1$, then for any choice of $\{\alpha_{k},\beta_{k}\}_{k=2}^{K}$, the average number of transmissions saved $K_{s}$ by ordering the global communications is bounded from below by

[TABLE]

where ${P_{D,k}({\delta^{\left(1\right)}})}$ and $P_{f,k}({\delta^{\left(0\right)}})$ were previously defined in (28) and (29), respectively.

Proof:

The proof of this theorem is omitted, since it follows the proof in [5]. The difference from [5] is that we need to employ ${P_{D,k}({\delta^{\left(1\right)}})}$ in (28) and $P_{f,k}({\delta^{\left(0\right)}})$ in (29) which are completely different for this new problem. The other difference from [5] is that we add zero as a choice in (32) since the average number of transmissions saved $K_{s}$ makes sense when it is non-negative in reality. ∎

The lower bound on the average number of transmissions saved by ordering the global communications as demonstrated in Theorem 2 is valid for any choice of $\{\alpha_{k},\beta_{k}\}_{k=2}^{K}$ and any positive definite matrix $\bf\Sigma$ corresponding to a decomposable GGM. According to the expression on the right-hand side of (32), the lower bound on the average number of transmissions saved by ordering the global communications depends on the entries of the covariance matrix ${\bf\Sigma}$, the parameters $\{\alpha_{k},\beta_{k}\}_{k=2}^{K}$ and the prior probabilities $\pi_{0}$ and $\pi_{1}$.

Define

[TABLE]

where $\lambda_{\min,k}$ is the minimum eigenvalue of ${\bf{\Sigma}}_{{{\cal C}_{k}}}$ and $k=1,2,...,K$. The following theorem describes a lower bound on the average number of transmissions saved by ordering the global communications when $\lambda_{\min}$ is large.

Theorem 3

Consider the ordered transmission approach described in Theorem 1 which employs (19) with

[TABLE]

and

[TABLE]

for all $k=2,3,...,K$ using any $\gamma$ which satisfies

[TABLE]

With a sufficiently large eigenvalue $\lambda_{\min}$ defined in (33) and $K>1$, the average number of transmissions saved $K_{s}$ by ordering the global communications is bounded from below by

[TABLE]

Proof:

Refer to Appendix B. ∎

Note that Theorem 3 states that the average number of transmissions saved by ordering the global communications increases at least as fast as linearly proportional to the number of clusters $K$ while achieving the same probability of error as the conventional centralized detection approach. There are uncountable choices of $\{(\alpha_{k},\beta_{k})\}_{k=2}^{K}$ to guarantee the lower bound where the average number of transmissions saved can be larger than half the number of clusters employed.

The minimum eigenvalue of ${\bf{\Sigma}}_{{{\cal C}_{k}}}$, denoted $\lambda_{\min,k}$, approaching infinity leads the Kullback-Leibler divergence to increase to infinity. Some other statistical distance measures which quantify the distance between two distributions can also be employed here and lead to the same conclusion, for example the Bhattacharyya distance. The Kullback-Leibler divergence ${D_{{\text{KL}}}}({f({{{\bf{x}}_{{{\cal C}_{k}}}}|{{H_{0}}}})\|{f({{{\bf{x}}_{{{\cal C}_{k}}}}|{{H_{1}}}})}})$ between $f({{{\bf{x}}_{{{\cal C}_{k}}}}|{{H_{0}}}})$ and $f({{{\bf{x}}_{{{\cal C}_{k}}}}|{{H_{1}}}})$ increases to infinity for all $k$, since

[TABLE]

where $M_{k}$ is the dimension of ${\bm{\Sigma}}_{{{\cal C}_{k}}}$ and $\lambda_{m,k}$ is the $m$-th eigenvalue of ${\bm{\Sigma}}_{{{\cal C}_{k}}}$. Thus, the fact that the significant average number of transmissions saved can be obtained as shown in Theorem 3 is consistent with our intuition that with a large distance between the cluster covariance matrices under two hypotheses, they are easy to tell apart so we do not need many observations to make a reliable decision. Note that $\lambda_{\min,k}\rightarrow\infty$ for all $k=1,2,...,K$ is only a sufficient condition rather than a necessary condition. Deriving necessary and sufficient conditions for saving half of the transmissions for the covariance matrix testing problem will be pursued in our future work.

IV Numerical Results

In this section, numerical examples of two representative classes of decomposable Gaussian graphical models with chain structure and tree structure are presented to illustrate the lower bounds on the number of transmissions saved by employing ordered transmissions.

IV-A Average Percentage of Transmissions Saved versus the Minimum Eigenvalue

In this subsection, the theoretical lower bound in (32) is compared with the average number of transmissions saved by ordering the CH to FC communications from Monte Carlo simulations. Consider a class of decomposable Gaussian graphical models with $20$ clusters as shown in Fig. 3. As depicted in Fig. 3, each cluster consists of $5$ sensors, and every two consecutive clusters are coupled through a $1$-sensor separator. In the simulation results in Fig. 4, we set $\pi_{0}=\pi_{1}=0.5$ and $\gamma=0.5/(2^{19}-1)$. Assume the eigenvalues of ${\bf\Sigma}_{{\cal C}_{1}}$ form a vector with five elements which are uniformly distributed from $\alpha$ to $1.5\alpha$ where $\alpha$ can be varied to change the minimum eigenvalue of ${\bf\Sigma}_{{\cal C}_{1}}$. We generate a diagonal matrix ${\bf\Lambda}$ with these eigenvalues along the diagonal and use a unitary matrix ${\bf V}$ to construct a non-diagonal matrix ${\bf\Sigma}_{{\cal C}_{1}}={\bf V}^{T}{\bf\Lambda}{\bf V}$ where the unitary matrix is generated by employing Gram -Schmidt orthonormalization [20] while ensuring (6) and (7) are satisfied. We take ${\bf\Sigma}_{{\cal C}_{k}}$ to be equal to ${\bf\Sigma}_{{\cal C}_{1}}$ for $k=2,3,...,K$. We use the same ${\bf\Sigma}_{{\cal C}_{k}}$ during all the Monte Carlo runs so the statistical descriptions of the observations are the same. However, the actual observations are randomly regenerated in each Monte Carlo run so they are different over each Monte Carlo run. We employ the Monte Carlo method (20000 runs) to obtain the average number of transmissions saved by ordering the global communications as illustrated in Fig. 4 for different values of $\alpha$. For comparison, in Fig. 4, the theoretical lower bound in (32) is also provided. It is shown that as the parameter $\alpha$ increases (the minimum eigenvalue $\lambda_{\min}$ increases), the value of the theoretical lower bound on the average number of transmissions saved by ordering the global communications increases. As expected from our analysis, the simulation results in Fig. 4 show that the lower bound on the average percentage of transmissions saved nearly equals to 0.45 when $a=199$ which is consistent with Theorem 3 since ${(\left\lceil{\frac{K}{2}}\right\rceil-1)/K}=0.45$ when $K=20$. These results also illustrate that the theoretical lower bound in Theorem 2 is also a valid lower bound on the simulated estimates for the average number of transmissions saved by employing ordered transmissions for the specific cases considered.

IV-B Average Number of Transmissions Saved versus the Number of Clusters

In this subsection, we investigate the average number of transmissions saved by ordering the global communications for a different number of clusters $K$. We consider the same class of decomposable Gaussian graphical models as in Fig. 3. In the simulation, we also set $\pi_{0}=\pi_{1}=0.5$ and $\gamma=0.5/(2^{K}-1)$ and generate ${\bf\Sigma}_{{\cal C}_{k}}$ for $k=1,2,...,K$ in the same way as shown in the previous subsection. Fig. 5 illustrates the average number of transmissions saved by ordering the global communications versus $K$ for different values of $\alpha$. For comparison, in Fig. 5, the limiting theoretical lower bound in (37) is also provided. Fig. 5 illustrates that the average number of transmissions saved by ordering the global communications increases approximately linearly with $K$ for every value of $\alpha$. It indicates that the rate of increase with $K$ becomes faster when the parameter $\alpha$ is increased. In addition, for the sufficiently large minimum eigenvalue $\alpha>1.4$ in this scenario, ${\left\lceil{\frac{K}{2}}\right\rceil-1}$ asymptotically serves as a lower bound on the number of transmissions saved by ordering the global communications.

Next, we consider a different class of decomposable Gaussian graphical model with the binary tree structure illustrated in Fig. 6 where each cluster has 4 nodes and each separator set has 1 node. We set $\pi_{0}=\pi_{1}=0.5$ and $\gamma=0.5/(2^{K}-1)$. The diagonal elements of ${\bf\Sigma}_{{\cal C}_{k}}$ are set to be $x^{2}$ and the other elements of ${\bf\Sigma}_{{\cal C}_{k}}$ are set to equal to $x/10$ where we change the minimum eigenvalue of ${\bf\Sigma}_{{\cal C}_{k}}$ by changing the parameter $x$ since the minimum eigenvalue of ${\bf\Sigma}_{{\cal C}_{k}}$ equals to $x^{2}-x/10$. We fix each cluster to have the same value of $x$ in each iteration. We consider the average number of transmissions saved versus the number of clusters for $x=1.1,1.2,1.4$ and $1.6$ which correspond to $\lambda_{\min,k}=1.10,1.32,1.82$ and $2.40$, respectively. Fig. 7 implies the average number of transmissions saved by ordering the global communications increases approximately linearly with $K$ for every value of $\lambda_{\min,k}$. The larger the value of $\lambda_{\min,k}$, the larger the slope of curves which is very similar to the result in Fig. 5.

V Conclusion

We have proposed the ordered transmission approach to testing the covariance matrix of a GGM where transmissions can be saved without performance loss. After each CH collects and summarizes the data from the sensor nodes, this approach is employed to reduce the number of transmissions from the CHs to the FC. A lower bound on the average number of transmissions was derived which can approach approximately half the number of clusters when the minimum eigenvalue of the covariance matrix under the alternative hypothesis in each cluster becomes sufficiently large.

Appendix A Details of (19)

Plugging (13) and (14) into (II-B), the test statistics $T(\bf x)$ can be rewritten as

[TABLE]

In going from (39) to (40), we group the separator terms involving the same observations in several ways since the separator set data is actually contained in the data for the associated clusters. Then (41) follows by using the definition in (20)–(24). In going from (41) to (42), we only employ the subvector of $\bf x$, denoted as ${\bf{x}}_{{{\cal C}_{K}}}$, which consists of the elements of $\bf x$ corresponding to the indices contained in the $k$-th cluster since other subvectors of $\bf x$ will not have an impact for this cluster.

Appendix B Proof of *Theorem 3 *

We can bound the eigenvalues of \big{(}{{\bf I}_{{{\cal C}_{k}}}^{-1}}-({\bf{\Sigma}}_{{{\cal C}_{k}}})^{-1}\big{)} for $k=1,2,...,K$, denoted as $eig\{{{\bf I}_{{{\cal C}_{k}}}^{-1}}-({\bf{\Sigma}}_{{{\cal C}_{k}}})^{-1}\}$, as follows

[TABLE]

where $\lambda_{\min,k}$ and $\lambda_{\max,k}$ are the minimum and maximum eigenvalues of ${\bf{\Sigma}}_{{{\cal C}_{k}}}$ respectively. Then we obtain

[TABLE]

By employing the Eigenvalue Interlacing Theorem [20] which mainly states that the eigenvalues of the principal submatrix can be bounded by eigenvalues of the whole matrix, we obtain for ${j\in{{\cal Q}_{k}}}$

[TABLE]

where $\lambda_{\min,k}$ and $\lambda_{\max,k}$ are the minimum and maximum eigenvalues of the matrix ${\bf{\Sigma}}_{{{\cal C}_{k}}}$. Similarly, for $k=2,3,...,K$, we obtain

[TABLE]

Then we can obtain

[TABLE]

for ${j\in{{\cal Q}_{k}}}$ and

[TABLE]

for $k=2,3,...,K$.

Weyl inequality (see Theorem 4.3.1 in [20] for more details) states that let $A$ and $B$ be $n\times n$ Hermitian and let the respective eigenvalues of $A$, $B$, and $A+B$ be $\{\lambda_{i}(A)\}_{i=1}^{n}$, $\{\lambda_{i}(B)\}_{i=1}^{n}$, and $\{\lambda_{i}(A+B)\}_{i=1}^{n}$, each algebraically nondecreasing ordered, then

[TABLE]

and

[TABLE]

for each $i=1,2,...,n$. By employing (50) and (51), we can obtain that for $i=1,2,...,n$,

[TABLE]

where $\lambda_{1}(A)$ and $\lambda_{1}(B)$ are the minimum eigenvalues of $A$ and $B$ respectively, and $\lambda_{n}(A)$ and $\lambda_{n}(B)$ are the maximum eigenvalues of $A$ and $B$ respectively. By employing the inequality in (52) into (21), we obtain

[TABLE]

Then by employing (45) and (48)–(53), we obtain when ${\lambda_{\min,k}}\rightarrow\infty$, for $k=2,3,...,K$,

[TABLE]

This implies

[TABLE]

and

[TABLE]

For $k=1$, by employing (45), (48) and ${\bf J}_{1}$ in (20), we obtain when ${\lambda_{\min,k}}\rightarrow\infty$,

[TABLE]

By employing the definition of ${\beta_{k}}$ in (35), it is clear that for all $k=2,3,...,K$,

[TABLE]

where the positive-valued $\gamma$ is defined in (36). Furthermore, we can obtain

[TABLE]

where (59) is a consequence of (11) and (58), and (60) is from (36). By employing (11), (34), (36) and (58), we obtain for all $j=2,3,...,K$,

[TABLE]

Thus, by employing (54)–(61), we can obtain for all $k=1,2,...,K$,

[TABLE]

We can rewrite ${{L_{k}}({{\bf{x}}_{{{\cal C}_{k}}}})}$ under $H_{1}$ for $k=1,2,...,K$ in (19) as follows

[TABLE]

where we use ${\bf{x}}_{{{\cal C}_{k}}}={\bf{W}}_{{{\cal C}_{k}}}{\bf{y}}_{{{\cal C}_{k}}}$ to obtain (63) with ${\bf{y}}_{{{\cal C}_{k}}}\sim\mathcal{N}(0,{\bf{I}}_{{{\cal C}_{k}}})$ and a known whitening matrix ${\bf{W}}_{{{\cal C}_{k}}}$ satisfying the condition ${\bf{W}}_{{{\cal C}_{k}}}{\bf{W}}_{{{\cal C}_{k}}}^{T}={\bf{\Sigma}}_{{\cal C}_{k}}$. Let ${\bf{\Sigma}}_{{\cal C}_{k}}={\bf{V}}_{{{\cal C}_{k}}}^{T}diag\{\lambda_{0,k},\lambda_{1,k},...,\lambda_{M_{k}-1,k}\}{\bf{V}}_{{{\cal C}_{k}}}$ be the eigenvalue/eigenvector decomposition for ${\bf{\Sigma}}_{{\cal C}_{k}}$, from (55), we can obtain for $k=2,3,...,K$,

[TABLE]

For ${\bf{\Sigma}}_{{\cal C}_{k}}$, the eigenvalues are defined as $\lambda_{0,k},\lambda_{1,k},...,\lambda_{M_{k}-1,k}$. In going from (65) to (66), we define $t_{m,{{\cal C}_{k}}}$ for $m=0,1,...,M_{k}-1$ as the elements of the vector ${\bf{V}}_{{{\cal C}_{k}}}{\bf y}_{{{\cal C}_{k}}}$. In going from (66) to (67), we drop positive terms. Note that for $k=2,3,...,K$,

[TABLE]

which implies under $H_{1}$, for $k=2,3,...,K$,

[TABLE]

By substituting (63) into (28), we obtain for any given $\delta^{(1)}$, the detection probability $P_{D,k}(\delta^{(1)})$ in the $k$-th cluster for any $k=2,...,K$ is

[TABLE]

In going from (70) to (71), we employ $e_{k}$ in (24) and divide $(\lambda_{\max,k}-1)$ on both sides for large $\lambda_{\max,k}>1$. In going from (71) to (72), $\delta^{(1)}/(\lambda_{\max,k}-1)\rightarrow 0$, and $\ln\det({\bf{\Sigma}}_{{{\cal C}_{k}}})/(\lambda_{\max,k}-1)\rightarrow 0$ since $\ln\det({\bf{\Sigma}}_{{{\cal C}_{k}}})$ equals to the logarithm of the product of $eig\{{\bf{\Sigma}}_{{{\cal C}_{k}}}\}$ which is also upper bounded by $\ln M_{k}\lambda_{\max,k}$. Similarly, by employing (46) and (47), we can obtain $\ln\det({\bf{\Sigma}}_{{{\cal S}_{k}}})/(\lambda_{\max,k}-1)\rightarrow 0$ and $\ln\det({\bf{\Sigma}}_{{{\cal S}_{j}}})/(\lambda_{\max,k}-1)\rightarrow 0$ which lead to (72). The result in (73) follows by employing (69).

For $k=1$, by employing similar steps as used in (64)–(66), we obtain

[TABLE]

Then similar to (69), we can obtain

[TABLE]

Thus, we obtain for any given $\delta^{(1)}$, the detection probability $P_{D,1}(\delta^{(1)})$ in the first cluster is

[TABLE]

where we employ ${e}_{1}$ in (23) to obtain (76). (78) is a consequence of (75). Then by employing (73) and (78) together, finally we can show that for all $k=1,2,...,K$

[TABLE]

Note that ${\lambda_{\min,k}\rightarrow+\infty}$ for all $k=1,2,...,K$ if and only if the minimum among them $\lambda_{\min}\rightarrow+\infty$. Finally, we obtain for all $k=1,2,...,K$

[TABLE]

Now we consider the case under $H_{0}$ where ${\bf{x}}_{{{\cal C}_{k}}}\sim\mathcal{N}(0,{\bf{I}}_{{{\cal C}_{k}}})$ for $k=1,2,...,K$. From (56), we obtain

[TABLE]

where we employ the second inequalities in (54) and (57) to obtain the first inequality in (81). The equality in (81) follows since ${x}_{{m,{\cal C}_{k}}}$ is the m-th element of the vector ${\bf x}_{{{\cal C}_{k}}}\sim\mathcal{N}(0,{\bf{I}}_{{{\cal C}_{k}}})$ under $H_{0}$.

Before we show $\lim_{\lambda_{\min,k}\rightarrow+\infty}P_{f,k}(\delta^{(0)})=0$ for $k=2,3,...,K$, we first let

[TABLE]

where ${{\bf{\Sigma}}_{{{\cal S}_{j}}}}$ is the covariance matrix of the separator set for $j\in{{\cal Q}_{k}}$. By employing the Schur complement [20], we obtain

[TABLE]

Since it is clear that $eig\{({\bf{\Sigma}}_{{{\cal C}_{k}}})^{-1}\}\rightarrow 0$ when $\lambda_{\min,k}\rightarrow+\infty$, then by employing the Eigenvalue Interlacing Theorem222The Eigenvalue Interlacing Theorem states that the eigenvalues of the principal submatrix $\bf B$ obtained by deleting both $i$-th row and $i$-th column of the matrix $\bf A$ for some values of $i$ can be bounded by the eigenvalues of the matrix $\bf A$ [20]., we obtain the eigenvalues of the principal submatrix eig\{\big{(}{{\bf{\Xi}}}_{k}-{{\bf{\Omega}}}_{k}({{\bf{\Sigma}}_{{{\cal S}_{j}}}})^{-1}{{\bf{\Omega}}}_{k}^{T}\big{)}^{-1}\} (which are bounded by $eig\{({\bf{\Sigma}}_{{{\cal C}_{k}}})^{-1}\}$) must approach zero. Thus $eig\{({{\bf{\Xi}}}_{k}-{{\bf{\Omega}}}_{k}({{\bf{\Sigma}}_{{{\cal S}_{j}}}})^{-1}{{\bf{\Omega}}}_{k}^{T})\}\rightarrow+\infty$. By employing a property of the Schur complement [20], we obtain when $\lambda_{\min,k}\rightarrow+\infty$

[TABLE]

(88) implies that when $\lambda_{\min,k}\rightarrow+\infty$,

[TABLE]

Similar to the steps taken from (84) to (88), we can also obtain when $\lambda_{\min,k}\rightarrow+\infty$,

[TABLE]

Next we will show $\lim_{\lambda_{\min,k}\rightarrow+\infty}P_{f,k}(\delta^{(0)})=0$ for $k=2,3,...,K$. By substituting ${{L_{k}}({{\bf{x}}_{{{\cal C}_{k}}}})}$ in (19) into (29), we obtain for any given $\delta^{(0)}$, the false alarm probability $P_{f,k}(\delta^{(0)})$ in the $k$-th cluster for any $k=2,...,K$ is

[TABLE]

In going from (91) to (92), we employ $e_{k}$ in (24) and the inequality in (81). In going from (92) to (93), we employ (61) to lower bound $\ln\det({\bf{\Sigma}}_{{{\cal C}_{k}}})$ by the sum of $\sum_{j\in{{\cal Q}_{k}}}{\beta_{j}}\ln\det({\bf{\Sigma}}_{{{\cal C}_{k}}})$ and $\alpha_{k}\ln\det({\bf{\Sigma}}_{{{\cal C}_{k}}})$. By employing the results in (89) and (90), \delta^{(0)}\!+\!\!{\sum_{j\in{{\cal Q}_{k}}}{\beta_{j}}}\cdot\big{(}\!\ln\det({\bf{\Sigma}}_{{{\cal C}_{k}}})-\ln\det({\bf{\Sigma}}_{{{\cal S}_{j}}})\big{)}+\alpha_{k}\cdot\big{(}\ln\det({\bf{\Sigma}}_{{{\cal C}_{k}}})-\ln\det({\bf{\Sigma}}_{{{\cal S}_{k}}})\big{)} in (93) approaches positive infinity under the limit. (95) follows since $\sum_{m=0}^{M_{k}-1}{x}_{{m,{\cal C}_{k}}}^{2}$ can not be larger than the positive infinity.

By substituting ${{L_{1}}\left({{{\bf{x}}_{{{\cal C}_{1}}}}}\right)}$ in (19) into (29) and then taking similar steps as those taken to go from (91) to (84), we can also show that

[TABLE]

for the first cluster. Then by using the fact that the false alarm probability $P_{f,k}(\delta^{(0)})$ is non-negative and employing (95) and (96) together, finally we can show that for all $k=1,2,...,K$,

[TABLE]

Again, ${\lambda_{\min,k}\rightarrow+\infty}$ for all $k=1,2,...,K$ if and only if $\lambda_{\min}\rightarrow+\infty$. Then we obtain for all $k=1,2,...,K$

[TABLE]

Using the results in (80) and (98), the limiting behavior of the lower bound on the average number of transmissions saved by ordering the global communications in (32) for $K>1$ becomes

[TABLE]