Insense: Incoherent Sensor Selection for Sparse Signals

TL;DR

Insense is a novel sensor selection algorithm designed for sparse signals, optimizing sensor subsets to enable accurate recovery from fewer measurements than the signal dimension, outperforming existing methods.

Contribution

The paper introduces Insense, a new sensor selection algorithm based on coherence minimization, effective for sparse signals and applicable where traditional methods fail.

Findings

Insense outperforms existing algorithms on six datasets.

Effective in reducing sensors needed for sparse signal recovery.

Validated on real-world applications like microbial diagnostics.

Abstract

Sensor selection refers to the problem of intelligently selecting a small subset of a collection of available sensors to reduce the sensing cost while preserving signal acquisition performance. The majority of sensor selection algorithms find the subset of sensors that best recovers an arbitrary signal from a number of linear measurements that is larger than the dimension of the signal. In this paper, we develop a new sensor selection algorithm for sparse (or near sparse) signals that finds a subset of sensors that best recovers such signals from a number of measurements that is much smaller than the dimension of the signal. Existing sensor selection algorithms cannot be applied in such situations. Our proposed Incoherent Sensor Selection (Insense) algorithm minimizes a coherence-based cost function that is adapted from recent results in sparse recovery theory. Using six datasets, including two real-world datasets on microbial diagnostics and structural health monitoring, we demonstrate the superior performance of Insense for sparse-signal sensor selection.

Tables

Table 1. TABLE I: Comparison of Insense against the baseline algorithms on selecting M = 10 𝑀 10 M=10 rows from a structured Identity/Gaussian Φ Φ \Phi . Insense selects the set of sensors with the smallest μ avg subscript 𝜇 avg \mu_{\text{avg}} and achieves the best BP recovery performance.

	${\mu }_{\text{avg}\left({\Phi }_{\Omega }\right)}$	FP(${\Phi }_{\Omega }$)	CN(${\Phi }_{\Omega }$)	BP accuracy %
Insense	0.3061 $\pm$ 0.0047	1019 $\pm 313$	1.93 $\pm$ 0.19	92.27 $\pm$ 1.42
FrameSense	–	0.00 $\pm$ 0.00	1.00 $\pm$ 0.00	4.00 $\pm$ 0.00
EigenMaps	–	0.00 $\pm$ 0.00	1.00 $\pm$ 0.00	4.00 $\pm$ 0.00
MSE-G	0.3872 $\pm$ 0.0305	1155 $\pm$ 374	11.51 $\pm$ 0.93	57.91 $\pm$ 1.09
MI-G	–	0.00 $\pm$ 0.00	1.00 $\pm$ 0.00	4.00 $\pm$ 0.00
Entropy-G	–	0.00 $\pm$ 0.00	1.00 $\pm$ 0.00	4.00 $\pm$ 0.00
Determinant-G	–	0.00 $\pm$ 0.00	1.00 $\pm$ 0.00	4.00 $\pm$ 0.00
Greedy SS	–	0.00 $\pm$ 0.00	1.00 $\pm$ 0.00	4.00 $\pm$ 0.00
Convex SS	0.3137 $\pm$ 0.0075	2279 $\pm$ 470	2.22 $\pm$ 0.25	88.64 $\pm$ 3.64

Table 2. TABLE II: Comparison of Insense against the baseline algorithms on selecting M = 10 𝑀 10 M=10 rows from a structured Uniform/Gaussian Φ Φ \Phi . Insense selects the set of sensors with the smallest μ avg subscript 𝜇 avg \mu_{\text{avg}} and achieves the best BP recovery performance.

	${\mu }_{\text{avg}}\left({\Phi }_{\Omega }\right)$	FP(${\Phi }_{\Omega }$)	CN(${\Phi }_{\Omega }$)	Gaussian sensor ratio %	BP accuracy %
Insense	0.3165 $\pm$ 0.0023	9320 $\pm 3292$	1.46 $\pm$ 0.07	100 $\pm$ 0	58.55 $\pm$ 2.64
FrameSense	0.3273 $\pm$ 0.0059	6095 $\pm$ 1708	3.19 $\pm$ 0.92	84 $\pm$ 5	58.15 $\pm$ 2.26
EigenMaps	0.3215 $\pm$ 0.0021	7230 $\pm$ 2319	2.07 $\pm$ 0.12	90 $\pm$ 0	57.60 $\pm$ 3.72
MSE-G	0.5805 $\pm$ 0.0440	78530 $\pm$ 12450	5.99 $\pm$ 0.31	17 $\pm$ 4	49.90 $\pm$ 3.54
MI-G	0.6814 $\pm$ 0.0556	93260 $\pm$ 109250	6.26 $\pm$ 0.77	7 $\pm$ 4	51.60 $\pm$ 5.21
Entropy-G	0.7007 $\pm$ 0.0804	98950 $\pm$ 16216	6.61 $\pm$ 0.48	5 $\pm$ 7	53.70 $\pm$ 5.21
Determinant-G	0.7303 $\pm$ 0.0545	105700 $\pm$ 11228	6.57 $\pm$ 0.31	3 $\pm$ 4	55.50 $\pm$ 4.50
Greedy SS	0.7303 $\pm$ 0.0545	105700 $\pm$ 11228	5.57 $\pm$ 0.31	3 $\pm$ 4	55.50 $\pm$ 4.50
Convex SS	0.5788 $\pm$ 0.1140	75270 $\pm$ 27383	5.97 $\pm$ 0.77	20 $\pm$ 15	54.40 $\pm$ 4.20

Table 3. TABLE III: Comparison of Insense against the baseline algorithms on selecting M 𝑀 M DNA probes to identify pathogenic samples containing K 𝐾 K bacterial organisms. Insense selects the sets of DNA probes that achieve the best pathogen identification performance.

	BP accuracy in detecting organisms %
Number of organisms	$K=2$					$K=3$				$K=5$
Number of probes ($M$)	$5$	$8$	$12$	$15$		$8$	$12$	$15$	$20$		$12$	$15$	$20$	$25$
Insense	25.52	68.33	94.78	99.65		26.46	71.74	93.95	99.53		16.78	51.95	92.71	99.10
FrameSense	27.73	61.83	88.40	95.71		22.70	62.32	82.29	98.36		10.79	35.16	81.92	96.50
EigenMaps	14.97	49.65	84.69	94.66		13.17	54.68	78.09	96.25		6.69	27.47	72.13	95.30
MSE-G	27.26	60.79	91.53	97.91		22.01	67.16	89.15	98.40		14.69	43.26	83.52	97.40
MI-G	26.22	59.98	89.68	96.40		20.96	65.69	84.10	97.39		13.49	37.96	79.72	96.00
Entropy-G	27.96	61.25	91.53	98.61		21.51	66.35	88.96	99.19		14.19	42.86	89.61	97.50
Determinant-G	14.85	46.75	82.13	94.55		12.49	48.97	76.13	96.03		6.29	24.48	72.73	92.81
Greedy SS	25.52	57.54	87.70	96.87		19.72	59.65	84.64	97.34		10.99	36.16	80.22	94.11
Convex SS	15.89	53.36	87.94	98.94		14.29	57.58	87.59	98.89		7.69	38.46	83.52	98.40
Random	25.57	61.53	88.79	96.66		22.37	62.29	86.15	97.72		12.79	38.88	82.94	86.44

Table 4. TABLE IV: Comparison of Insense against the baseline algorithms on sampling M 𝑀 M time instances that best localize cracks on a plane in a structural health monitoring problem. Insense selects the set of sensors with the smallest μ avg subscript 𝜇 avg \mu_{\text{avg}} and achieves the best damage localization performance.

	${\mu }_{\text{avg}}\left({\Phi }_{\Omega }\right)$							BP accuracy in localizing cracks %							Running time (s)
Number of samples ($M$)	$5$	$6$	$7$	$8$	$9$	$10$		$5$	$6$	$7$	$8$	$9$	$10$		$10$
Insense	0.2491	0.2251	0.2185	0.2127	0.2075	0.2030		40.99	60.49	76.49	86.09	91.39	94.64		4.12
FrameSense	–	0.4051	0.3843	0.3179	0.2717	0.2640		24.41	51.58	63.73	81.92	89.04	91.14		0.18
EigenMaps	0.4341	0.4102	0.4028	0.3950	0.3841	0.3699		25.13	41.16	40.55	51.38	64.52	80.45		0.11
MSE-G	0.9832	0.9779	0.9752	0.9748	0.9707	0.9637		39.50	53.50	63.73	79.18	86.28	93.50		0.97
Entropy-G	0.9694	0.9649	0.9615	0.9568	0.9525	0.9507		40.89	58.95	71.35	80.98	86.50	92.36		0.16
Greedy SS	–	–	–	–	–	–		9.32	14.96	19.55	28.36	45.77	57.43		1.65
Convex SS	0.9702	0.9685	0.9611	0.9571	0.9519	0.9451		38.92	59.56	72.42	82.83	89.98	94.56		4.52
Random	0.7980	0.7976	0.8083	0.8147	0.8111	0.8132		33.84	52.20	67.51	79.60	88.19	93.75		< ${10}^{-4}$

Equations

$$y_{\Omega }={\Phi }_{\Omega }x$$

$${\mu }_{\text{avg}}^2({\Phi }_{\Omega })=\frac{1}{\left(\genfrac{}{}{0ex}{}{N}{2}\right)}\sum _{1\le i<j\le N}{\mu }_{ij}^2({\Phi }_{\Omega }).$$

$$\text{FP}({\Phi }_{\Omega })=\sum _{\forall (i,j)\in \Omega ,i<j}|⟨{\varphi }^i,{\varphi }^j⟩{|}^2,$$

$${\mu }_{\text{avg}}^2({\Phi }_{\Omega })=\frac{1}{\left(\genfrac{}{}{0ex}{}{N}{2}\right)}\sum _{1\le i<j\le N}\frac{|⟨{\varphi }_i,{\varphi }_j⟩{|}^2}{\Vert {\varphi }_i{\Vert }^2\Vert {\varphi }_j{\Vert }^2},$$

$$\underset{z\in \{0,1{\}}^D}{\mathrm{minimize}}$$

$$G={\Phi }^{T}Z\Phi ,1^{T}z=M,$$

$$\underset{z\in [0,1{]}^D}{\mathrm{minimize}}$$

$$G={\Phi }^{T}Z\Phi ,1^{T}z=M,$$

$$f_{ϵ}(z)=\sum _{1\le i<j\le N}\frac{{G_{ij}}^2+{ϵ}_1}{G_{ii}{G}_{jj}+{ϵ}_2}\text{where}G={\Phi }^{T}Z\Phi ,$$

$$({\nabla }_{z}f{)}_i={\left(\Phi {\nabla }_{G}f{\Phi }^T\right)}_{ii}\text{for}i=1,\dots ,D.$$

$$({\nabla }_{G}f{)}_{ij}=\{\begin{array}{ll}\frac{2G_{ij}}{G_{ii}{G}_{jj}+{ϵ}_2},& i<j\\ -\sum _{\forall \ell \ne i}\frac{G_{\ell \ell }\left(G_{i\ell }^2+{ϵ}_1\right)}{{\left(G_{ii}{G}_{\ell \ell }+{ϵ}_2\right)}^2},& i=j\\ 0,& \text{elsewhere.}\end{array}$$

$$z^{k+1}=P_{\text{SBS}}\left(z^k-{\gamma }^k{\nabla }_{z}f(z^k)\right),$$

$$\underset{z}{\mathrm{minimize}}\frac{1}{2}\Vert z-y{\Vert }_2^2,$$

$$\text{subject to}\sum _i{z}_i=M\text{and}{z}_i\in [0,1]\forall i=1,\dots ,D.$$

$$f(z,\lambda ,\alpha ,\beta )=\frac{1}{2}\Vert z-y{\Vert }_2^2$$

$$+\lambda \left(\sum _i{z}_i-M\right)+\sum _i{\alpha }_i(-z_i)+\sum _i{\beta }_i(z_i-1),$$

$$z_i-y_i+\lambda -{\alpha }_i+{\beta }_i=0,\forall i,$$

$$\sum _i{z}_i-M=0,$$

$${\alpha }_i(-z_i)=0,{\beta }_i(z_i-1)=0,{\alpha }_i,{\beta }_i\ge 0,0\le z_i\le 1,\forall i.$$

$$\lambda =\frac{M-K_1-{\sum }_{i\in \zeta }{y}_i}{|\zeta |},$$

Full text

I Introduction

The accelerating demand for capturing signals at high resolution is driving acquisition systems to employ an increasingly large number of sensing units. However, factors like manufacturing costs, physical limitations, and energy constraints typically define a budget on the total number of sensors that can be implemented in a given system. This budget constraint motivates the design of sensor selection algorithms [1] that intelligently select a subset of sensors from a pool of available sensors in order to lower the sensing cost with only a small deterioration in acquisition performance.

In this paper, we extend the classical sensor selection setup, where $D$ available sensors obtain linear measurements of a signal $x\in\mathbb{R}^{N}$ according to $y=\Phi x$ with each row of $\Phi\in\mathbb{R}^{D\times N}$ corresponding to one sensor. In this setup, the sensor selection problem is one of finding a subset $\Omega$ of sensors (i.e., rows of $\Phi$) of size $|\Omega|=M$ such that the signal $x$ can be recovered from its $M$ linear measurements

[TABLE]

with minimal reconstruction error. Here, $\Phi_{\Omega}\in\mathbb{R}^{M\times N}$ is called the sensing matrix; it contains the rows of $\Phi$ indexed by $\Omega$.

The lion’s share of current sensor selection algorithms [1, 2, 3] select sensors that best recover an arbitrary signal $x$ from $M>N$ measurements. In this case, (1) is overdetermined. Given a subset of sensors $\Omega$, the signal $x$ is recovered simply by inverting the sensing matrix while computing $\Phi_{\Omega}^{\dagger}y_{\Omega}$, where $\Phi_{\Omega}^{\dagger}$ is the pseudoinverse of $\Phi_{\Omega}$.

Such approaches do not exploit the fact that many real-world signals are (near) sparse in some basis [4]. It is now well-known that (near) sparse signals can be accurately recovered from a number of linear measurements $M\ll N$ using sparse recovery/compressive sensing (CS) techniques [5, 6, 7]. Conventional sensor selection algorithms are not designed to exploit low-dimensional signal structure. Indeed, they typically fail to select the appropriate sensors for sparse signals in this underdetermined setting ($M<N$).

In this paper, we develop a new sensor selection framework that finds the optimal subset of sensors $\Omega$ that best recovers a (near) sparse signal $x$ from $M<N$ linear measurements (see Fig. 1). In contrast to the conventional sensor selection setting, here the sensing equation (1) is underdetermined, and it can not be simply inverted in closed form.

A key challenge in sensor selection in the underdetermined setting is that we must replace the cost function that has been so useful in the classical, overdetermined setting, namely the estimation error $\|x-\hat{x}\|_{2}^{2}$ (or the covariance of the estimation error in the presence of noise). In the overdetermined setting, this error can be obtained in closed form simply by inverting (1). In the underdetermined setting, this error has no closed form expression. Indeed, recovery of a sparse vector $x$ from $y_{\Omega}$ requires a computational scheme [8, 9].

Fortunately, the sparse recovery theory tells us that one can reliably recover a sufficiently sparse vector $x$ from its linear measurements $y_{\Omega}$ when the columns of the sensing matrix $\Phi_{\Omega}$ are sufficiently incoherent [10, 11, 12]. Define the coherence between the columns $\phi_{i}$ and $\phi_{j}$ in the sensing matrix $\Phi_{\Omega}$ as $\mu_{ij}(\Phi_{\Omega})=\frac{|\langle\phi_{i},\phi_{j}\rangle|}{\|\phi_{i}\|\|\phi_{j}\|}$. If the values of $\mu_{ij}(\Phi_{\Omega})$ for all pairs of columns $(i,j)$ are bounded by a certain threshold, then sparse recovery algorithms such as Basis Pursuit (BP) [13, 14, 10] can recover the sparse signal $x$ exactly. This theory suggests a new cost function for sensor selection. To select the sensors $\Omega$ that reliably recover a sparse vector, we can minimize the average squared coherence

[TABLE]

The challenge now becomes formulating an optimization algorithm that selects the subset of the rows of $\Phi$ (the sensors) whose columns have the smallest average squared coherence.

I-A Contributions

We make three distinct contributions in this work. First, we are the first to propose and study the sparse-signal sensor selection problem. Below we demonstrate that the standard cost functions used in overdetermined sensor selection algorithms are not suitable for the underdetermined case.

Second, we develop a new sensor selection algorithm that optimizes the new cost function (2); call it the Incoherent Sensor Selection (Insense) algorithm. Insense employs an efficient optimization technique to find a subset of sensors with smallest average coherence among the columns of the selected sensing matrix $\Phi_{\Omega}$. The optimization technique – projection onto the convex set defined by a scaled-boxed simplex (SBS) constraint – is of independent interest. We have made the codes for the Insense algorithm available online at https://github.com/amirmohan/Insense.git.

Third, we demonstrate the superior performance of Insense over conventional sensor selection algorithms using an exhaustive set of experimental evaluations that include real-world datasets from microbial diagnostics and structural health monitoring and six performance metrics: average mutual coherence, maximum mutual coherence, sparse recovery performance, frame potential, condition number, and running time. We demonstrate that, for the kinds of redundant, coherent, or structured $\Phi$ that are common in real-world applications, Insense finds the best subset of sensors in terms of sparse recovery performance by a wide margin. Indeed, in these cases, many conventional sensor selection algorithms fail completely.

I-B Paper organization

We first overview the state-of-the-art overdetermined sensor selection algorithms and elaborate on the unique properties of the undetermined problem and its relation to CS in Section II. Section III states our coherence minimization formulation to solve the underdetermined sensor selection problem, and Section IV details the Insense algorithm. Section V presents our simulation results on synthetic and real-world datasets, and Section VI concludes the paper.

II Related work

This section overviews the conventional sensor selection algorithms and their cost functions for the overdetermined case ($M>N$). We then detail the relationship of the underdetermined case to the field of CS.

II-A Conventional sensor selection algorithms

Existing sensor selection algorithms mainly study the sensor selection problem in the overdetermined regime (when $M\geq N$) [1, 2, 15, 3].

In the overdetermined regime, robust signal recovery can be obtained using the solution to the least squares (LS) problem in the sensing model (1), which motivates as a cost function the mean squared error (MSE) [16, 17, 18] or a proxy of the MSE [19, 20, 21] of the LS solution.

For instance, Joshi, et al. [1] employ a convex optimization-based algorithm to minimize the log-volume of the confidence ellipsoid around the LS solution of $x$. Shamaiah et al. [2] develop a greedy algorithm that outperforms the convex approach in terms of MSE. FrameSense [3] minimize the frame potential (FP) of the selected matrix

[TABLE]

where $\phi^{i}$ represents the $i^{\text{th}}$ row of $\Phi$. Several additional sensor selection algorithms that assume a non-linear observation model [22, 23] also operate only in the overdetermined regime.

II-B Connections to compressive sensing

Our model for sensor selection has strong connections to, and enables powerful extensions of, the CS problem, in which a (near) sparse signal is recovered from a small number of randomized linear measurements [5, 6, 7]. First, note that CS theory typically employs random sensing matrices; for instance it has been shown that many ensembles of random matrices, including partial Fourier, Bernoulli, and Gaussian matrices, result in sensing matrices with guaranteed sparse recovery[24, 25]. Recently, there have been efforts to design sensing matrices that outperform random matrices for certain recovery tasks [26, 27, 28, 29, 30]. For instance, Grassmannian matrices [27] attain the smallest possible mutual coherence and hence can lead to better performance in some applications.

However, many real-world applications do not involve random or Grassmannian sensing matrices; rather the sensing matrix is dictated by physical constraints that are specific and unique to each application. For example, in the sparse microbial diagnostic problem [31], the entries of the sensing matrix $\Phi$ are determined by the hybridization affinity of random DNA probes to microbial genomes and do not necessarily follow a random distribution. Similarly, in the structural health monitoring problem [32, 33, 34], the sensing matrix is constrained by the solution to the wave equation. A key outcome of this paper is a new approach to construct practical and realizable sensing matrices using underdetermined sensor selection (via Insense).

III Problem statement

Consider a set of $D$ sensors taking nonadaptive, linear measurements of a $K$-sparse (i.e., with $K$ non-zero elements) vector $x\in\mathbb{R}^{N}$ following the linear system $y=\Phi x$, where $\Phi\in\mathbb{R}^{D\times N}$ is a given sensing matrix. We aim to select a subset $\Omega$ of sensors of size $|\Omega|=M\ll D$, such that the sparse vector $x$ can be recovered from the resulting $M<N$ linear measurements $y_{\Omega}=\Phi_{\Omega}x$ with minimal reconstruction error. Here, $\Phi_{\Omega}$ contains the rows of $\Phi$ indexed by $\Omega$, and $y_{\Omega}$ contains the entries of $y$ indexed by $\Omega$. This model for the sensor selection problem can be adapted to more general scenarios. For example, if the signal is sparse in a basis $\Psi$, then we simply consider $\Phi=\Theta\Psi$ as the new sensing matrix, where $\Theta$ is the original sensing matrix.

In order to find a subset $\Omega$ of sensors (rows of $\Phi$) that best recovers a sparse signal $x$ from $y_{\Omega}$,111Or find one of the solutions if many solutions exists. we aim to select a submatrix $\Phi_{\Omega}\in\mathbb{R}^{M\times N}$ that attains the lowest average squared coherence

[TABLE]

where $\phi_{i}$ denotes the $i^{\text{th}}$ column of $\Phi_{\Omega}$. The term $\mu_{\text{avg}}$ averages the off-diagonal entries of $\Phi_{\Omega}^{T}\Phi_{\Omega}$ (indexed by $1\leq i<j\leq N$) after column normalization. Other measures of coherence (e.g., max coherence $\mu_{\text{max}}(\Phi_{\Omega})=\underset{i<j}{\max}{\mu_{ij}}$) can also be employed by slightly modifying the optimization procedure developed below. We choose to work with average coherence due to its simplicity and the fact that our experiments show that its performance is comparable to max coherence.

Define the diagonal selector matrix $Z={\text{diag}}(z)$ with $z=[z_{1},z_{2},z_{3},\dots,z_{D}]^{T}$ and $z_{i}\in\{0,1\}$, where $z_{i}=1$ indicates that the $i^{\text{th}}$ row (sensor) in $\Phi$ is selected and $z_{i}=0$ otherwise. This enables us to formulate the sensor selection problem as the following optimization problem

[TABLE]

where $\mathbf{1}$ is the all-ones vector. This Boolean optimization problem is combinatorial, since one needs to search over $D\choose M$ combinations of sensors to find the optimal set $\Omega$.

To overcome this complexity, we relax the Boolean constraint on $z_{i}$ to the box constraint $z_{i}\in[0,1]$ to arrive at the following problem

[TABLE]

which supports an efficient gradient–projection algorithm to find an approximate solution. We develop this algorithm next.

IV The Insense algorithm

We now outline the steps that Insense takes to solve the problem (6). We slightly modify the objective of (6) to

[TABLE]

where the small positive constants $\epsilon_{2}<\epsilon_{1}\ll 1$ make the objective well-defined and bounded over $z\in[0,1]^{D}$.

The objective function (7) is smooth and differentiable but non-convex; the box constraints on $z$ are linear. We minimize the objective using the iterative gradient-projection algorithm outlined in Alg. 1. The gradient $\nabla_{z}f\in\mathbb{R}^{D}$ can be computed using the multidimensional chain rule of derivatives [35] as

[TABLE]

The $N\times N$ upper triangular matrix $\nabla_{G}f$ is the gradient of $f$ in terms of the (auxillary) variable $G$ at the point $G=\Phi^{T}Z\Phi$, given by

[TABLE]

The Insense algorithm (Alg. 1) proceeds as follows. First, the variables $G$ and $Z$ are initialized. Next, we perform the following update in iteration $k$,

[TABLE]

where $P_{\text{SBS}}$ denotes the projection onto the convex set defined by the scaled boxed-simplex (SBS) constraints $\mathbf{1}^{T}z=M$ and $z\in[0,1]^{D}$. For certain bounded step size rules (e.g., $\gamma^{k}\leq 1/L$, where $L$ is the Lipschitz constant of $\nabla_{z}f$), the sequence $\{z^{k}\}$ generated by \frefeq:incsence seq converges to a critical point of the non-convex problem [36, 37]. In our implementation, we use a backtracking line search to determine $\gamma^{k}$ in each step [37].

IV-A The SBS projection

We now detail our approach to solving the SBS projection problem

[TABLE]

We emphasize that, for $M>1$, the SBS projection problem is significantly different from the (scaled-)simplex constraint $\left(\sum_{i}z_{i}=M\right)$ projection problem that has been studied in the literature [38, 39, 40], due to the additional box constraints $z_{i}\in[0,1]$.

The Lagrangian of the problem (IV-A) can be written as

[TABLE]

where $\lambda,{\alpha},{\beta}$ are Lagrange multipliers for the equality and inequality constraints, respectively. The Karush-Kuhn-Tucker (KKT) conditions are given by

[TABLE]

According to the complimentary slackness condition for the box constraint $z_{i}\in[0,1]$, we have the following three cases for $x_{i}$:

• (a)

$z_{i}=0$: $\beta_{i}=0,\alpha_{i}=y_{i}+\lambda>0$,

• (b)

$z_{i}=1$: $\alpha_{i}=0,\beta_{i}=1-y_{i}-\lambda>0$,

• (c)

$z_{i}\in(0,1)$: $\alpha_{i}=\beta_{i}=0,z_{i}=y_{i}+\lambda$.

Therefore, the value of $\lambda$ holds the key to the proximal problem \frefeq:prox. However, finding $\lambda$ is not an easy task, since we do not know which entries of $z$ will fall on the boundary of the box constraint (and are equal to either [math] or $1$).

In order to find the entries $z_{i}$ that are equal to [math], we assume without loss of generality that the values of $y$ are sorted in ascending order: $y_{1}\leq y_{2}\leq\ldots y_{D}$. We note that, in all three cases above, $z_{i}=\max(\min(y_{i}+\lambda,1),0)$. Therefore, $\sum_{i}z_{i}$ is a non-decreasing function of $\lambda$. We evaluate $\sum_{i}z_{i}$ at the following values of $\lambda$:

• •

$\lambda=-y_{1}$: $z_{1}=0,z_{i}=\min(y_{i}-y_{1},1)$ for $i\geq 2$,

• •

$\lambda=-y_{2}$: $z_{1}=z_{2}=0$, $z_{i}=\min(y_{i}-y_{1},1)$ for $i\geq 3$,

…

• •

$\lambda=-y_{D}$: $z_{1}=z_{2}=\ldots=z_{D}=0$.

Thus, the entries in $z$ that are equal to [math] correspond to the first $K_{0}$ entries in $y$, where $K_{0}$ is the largest integer $k$ such that $\sum_{i}\max(\min(y_{i}-y_{k},1),0)\geq M$.

Similarly, we can find the entries in $z$ that are equal to $1$ by negating $z$ and $y$ in \frefeq:prox. Let $p=-y$ and assume that its entries are sorted in ascending order; a procedure similar to that above shows that the entries in $z$ that are equal to $1$ correspond to the first $K_{1}$ entries in $p$, where $K_{1}$ is the largest integer $K$ such that $\sum_{i}\max(\min(p_{i}-p_{k}-1,0),-1)\geq-M$.

Knowing which entries in $z$ are equal to [math] and $1$, we can solve for the value of $\lambda$ by working with the entries with values in $(0,1)$. Using case (c) above and denoting the index set of these entries by $\zeta$, we have

[TABLE]

and the solution to \frefeq:prox is given by $z_{i}=\max(\min(y_{i}+\lambda,1),0)$.

V Experiments

In this section, we experimentally validate Insense (Alg. 1) using a range of synthetic and real-world datasets. In all experiments, we set $\epsilon_{1}=10^{-9}$ and $\epsilon_{2}=10^{-10}$ (anything in the range $\epsilon_{2}<\epsilon_{1}\ll 1$ can be utilized). We terminate Insense when the relative change of the cost function $\mu_{\text{avg}}^{2}(\Phi_{\Omega})$ drops below $10^{-7}$.

V-A Baselines and performance metrics

We compare Insense with several leading sensor selection algorithms, including Convex Sensor Selection [1], Greedy Sensor Selection [2], EigenMaps [15], and FrameSense [3]. We also compare with four greedy sensor selection algorithms that were featured in [3]. The first three minimize different information theoretic measures of the selected sensing matrix as a proxy to the MSE: the determinant in Determinant-G [19], mutual information (MI) in MI-G [20], and entropy in Entropy-G [21]. The final greedy algorithm, MSE-G [16, 17, 18], directly minimizes the MSE of the LS reconstruction error. The code for these baseline greedy algorithms were obtained from https://github.com/jranieri/OptimalSensorPlacement. We also compare with Random, a simple baseline that selects sensors at random.

We compare the sensor selection algorithms using the following six metrics:

• •

Average coherence $\mu_{\text{avg}}(\Phi_{\Omega})$.

• •

Maximum coherence $\mu_{\text{max}}(\Phi_{\Omega})$.

• •

Frame potential FP($\Phi_{\Omega}$) (see (3))

• •

Condition number CN($\Phi_{\Omega}$).

• •

BP recovery accuracy.

• •

Running time

Depending on the task, in some experiments we only report a subset of the metrics. To compute BP recovery accuracy, we average the performance of the BP algorithm [41] over multiple trials. In each trial we first generate a $K$-sparse vector $x$ whose non-zero entries are equal to one. We then use BP to recover $x$ from linear, nonadaptive (noiseless) measurements $y=\Phi_{\Omega}x$.

We repeat the same experiment for all $N\choose K$ sparse vectors $x$ with different support sets and report the percentage of trials that $x$ has been exactly recovered. When $N\choose K$ is too large (here, greater than 10,000), we run the BP algorithm on a smaller random subset of all $N\choose K$ sparse vectors $x$.

V-B Unstructured synthetic datasets

We first test the sensor section algorithms by applying them to random matrices. It is easy to show that asymptotically (i.e., when $N\rightarrow\infty$) random matrices do not favor certain rows (sensors) over others. In the non-asymptotic regime (i.e., when $N$ is finite) the choice of sensors for sparse recovery might be critical, since the probability that certain sets of sensors significantly outperform others increases. In this case, the sensor selection algorithm aims to identify these high-performing sensors. We generate three types of random sensing matrices $\Phi$ whose entries are drawn from Gaussian, Uniform, and Bernoulli distributions and compare the performance of Insense against the other baselines.

V-B1 Random Gaussian matrix

We conduct this experiment for 20 random trials and generate $100\times 100$ matrices $\Phi$ whose entries are independently drawn from a standard normal distribution. We use Insense and other baseline algorithms to select $M\in\{5,6,7,\ldots,30\}$ sensors. In Fig. 2(a,b) we report $\mu_{\text{avg}}$ and $\mu_{\max}$ of the selected sub-matrices $\Phi_{\Omega}$, with $|\Omega|=M$. All of the sensor selection algorithms have comparable performance to the random sensor selection strategy (Random), illustrating that only small improvements to the maximum and average coherence can be achieved using these algorithms.

V-B2 Random Uniform matrix

We repeat the previous experiment with a sensing matrix $\Phi$ whose entries are drawn uniformly at random from $[0,1]$. Fig. 2(c)) shows that Insense outperforms most of the baseline algorithms, including Random and Convex SS, in terms of $\mu_{\text{avg}}$. Despite selecting completely different sensors, FrameSense and EigenMaps have comparable performance to Insense in minimizing $\mu_{\text{avg}}$. Fig. 2(d) makes apparent the gap in maximum coherence $\mu_{\text{max}}$ between that achieved by Insense and the other baselines.

V-B3 Random Bernoulli matrix

We repeat the previous experiment with a sensing matrix $\Phi$ whose entries are 0 or 1 with equal probability.222In our experiments, the coherence minimization performance of all of the sensor selection algorithms was similar for Bernoulli ($1$/-$1$) matrices and Gaussian matrices. Fig. 2(e) shows that FrameSense, EigenMaps, and Insense have similar performance and outperform the other algorithms by a large margin in terms of average coherence $\mu_{\text{avg}}$.333When a selected matrices $\Phi_{\Omega}$ contains one column with all zero entries, the average coherence $\mu_{\text{avg}}$ is not defined. The missing values in some curves correspond to these instances. Fig. 2(f) shows a clear gap between Insense and the other baselines in terms of the maximum coherence $\mu_{\text{max}}$.

In summary, Insense selects reliable sensors that are consistently better than or comparable to the other baseline algorithms on random sensing matrices. This suggests that Insense could find application in designing sensing matrices that outperform random matrices for CS recovery tasks.

V-C Highly structured synthetic datasets

In contrast to random matrices, the sensing matrices in real-world applications often have imposed structures or redundancies. In such cases, careful sensor selection can mean the difference between low and high performance. We now explore sensor selection with structured over-complete matrices by constructing two synthetic datasets that resembles the redundancies and structures in real-world datasets. Similar over-complete basis has been explored in [41].

V-C1 Identity/Gaussian matrix

We construct our first highly structured dataset by concatenating two $50\times 50$ matrices: An identity matrix and a random matrix with i.i.d. Gaussian entries. Such matrices feature prominently in certain real-world CS problems.

For instance, in the universal DNA-based microbial diagnostics platform studied in [31], the identity and Gaussian matrices symbolize two different types of sensors: the identity matrix corresponds to a set of sensors that are designed to be specific to a single microbial target (column) in the dictionary $\Phi$, while the Gaussian matrix corresponds to a set of sensors that are universal for microbial targets in the dictionary. (We extend this simulation example with real experimental data below in Section V-D1.) As in [31], consider a bacterial detection scenario where the solution to the sparse recovery problem both detects and identifies the bacterial targets in a sample (through the support of the sparse vector $x$); here the goal is to maximize the average sparse recovery (detection) performance. On the one hand, if all of the sensors are selected from the identity submatrix, then nearly all of the selected sensors will lie dormant when detecting a particular bacterial target. On the other hand, if the sensors are selected from the Gaussian submatrix, then the selected sensors will work jointly to detect all bacterial targets, which provides both universality and better average sparse recovery performance [31]. To achieve a better sparse recovery performance, the sensor selection algorithm should select rows (sensors) from the Gaussian submatrix rather than the identity submatrix.

Table I compares the performance of Insense to the baseline algorithms for the problem of selecting $M=10$ rows from the structured Identity/Gaussian $\Phi$. We repeat the same experiment 10 times with different random Gaussian matrices.444Dashes correspond to instances where the selected matrices $\Phi_{\Omega}$ contain columns with all zero entries; here the average coherence $\mu_{\text{avg}}$ is undefined. In particular, Insense, Convex SS, and MSE-G are the only algorithms that select rows of the Gaussian sub-matrix. While achieving the minimum FP($\Phi_{\Omega}$) ($=0$), the other algorithms perform poorly on BP recovery. The greedy algorithms select rows from the identity matrix that result in columns with all-zero entries and thus fail to recover most of the entries in $x$.  Digging deeper, Insense selects rows with smaller column coherence than Convex SS and MSE-G. As a result, Insense achieves the best BP recovery performance (Table I) among these three algorithms.

In summary, this example demonstrates that minimizing a similarity metric imposed on the rows of the sensing matrix (such as frame potential, etc.) will not maximize the recovery performance of sparse signals. Our results also provide reassurance that the coherence among the columns of the sensing matrix is a useful performance objective.

V-C2 Uniform/Gaussian matrix

To study the quality of the box constraint relaxation in (6), we compare Insense against the baseline algorithms for a matrix $\Phi$ where we know the globally optimal index set of rows (sensors) $\Omega$. (For arbitrary $\Phi$, global combinatorial optimization is computationally intractable when $D,N>200$ or so.)

We concatenate a $10\times 200$ matrix with i.i.d. Gaussian entries and a $190\times 200$ matrix with i.i.d. $[0,1]$ uniform distribution entries. In this case, one would expect that the Gaussian submatrix has the lowest $\mu_{\text{avg}}$ when we set $M=10$. Fig. 3 visualizes the results of running Insense and the Convex SS baseline algorithm on such a $\Phi$. In all 10 random trials, Insense successfully selects all Gaussian rows and hence find the globally optimal set of sensors. FrameSense and EigenMaps miss, on average, 10–20% of the Gaussian sensors. The other baselines algorithms, including Convex SS, select only a small portion ($<$20%) of the Gaussian rows (sensors). Table II also indicates that Insense achieves better BP recovery performance, since it selects exclusively Gaussian rows, resulting in the minimum average coherence $\mu_{\text{avg}}$ of the resulting sensing matrix.

V-D Real-world datasets

Finally, we assess the performance of Insense on two real-world datasets from microbial diagnostics and structural health monitoring.

V-D1 Microbial diagnostics

Microbial diagnostics seek to detect and identify microbial organisms in a sample. Next-generation systems detect and classify organisms using DNA probes that bind (hybridize) to the target sequence and emit some kind of signal (e.g., fluorescence). Designing DNA probes for microbial diagnostics is an important application of sensor selection in the underdetermined sensing regime. For example, in the universal microbial sensing (UMD) framework [31], DNA probes acquire linear measurements from a microbial sample (e.g., bacterial, viral, etc.) in the form of a fluorescence resonance energy transfer (FRET) signal that indicates the hybridization affinity of a particular DNA probe to the organisms present in the sample. Given a matrix $\Phi$ that relates the hybridization affinities of DNA probes to microbial species, the objective is to recover a sparse vector $x$ comprising the concentrations of the organisms in the sample from as few linear measurements as possible.

We run Insense and the baseline sensor selection algorithms on a large sensing matrix comprising the hybridization affinity of $D=100$ random DNA probes to $N=42$ bacterial species (as described in [31]). For each algorithm, after selecting $M$ probes and constructing a sensing matrix $\Phi_{\Omega}$ with $|\Omega|=M$, we perform BP recovery for multiple sparse vectors $x$ with random support (corresponding to the presence of a random subset of bacterial organisms). We repeat the same experiment for all $\binom{N}{K}$ sparse vectors $x$ with $K=\{2,3,5\}$ non-zero elements (i.e., bacteria present) and report the average BP recovery performance on identifying the composition of the samples in Table III. (To report the BP recovery for $K=5$ organisms, we randomly generate 1000 realizations of the sparse vector $x$ with $5$ active elements and average the BP recovery performance on selected samples.)

The DNA probes selected by Insense outperform all of the baseline algorithms in identifying the bacterial organisms present. Specifically, Insense requires a smaller number of DNA probes than the other algorithm to achieve almost perfect detection performance (BP accuracy $>99\%$), suggesting that Insense is the most cost-efficient algorithm to select DNA probes for this application. Moreover, the performance gap between Insense and the other algorithms grows as the number of bacterial species present in the sample $K$ increases, indicating that Insense has better recovery performance in complex biological samples.

V-D2 Structural health monitoring

Structural health monitoring (SHM) studies the problem of detecting, localizing, and characterizing damage (e.g., cracks, holes, etc.) occurring on or inside structures such as buildings, bridges, pipes, etc. [32]. Typical SHM systems continuously monitor a target structure by transmitting signals that propagate throughout the structure and then analyzing the reflected signals measured using an array of sensors. Classical methods for damage localization calculate the time-of-flight (TOF) and estimate the location of a crack using triangulation, which is computationally intensive.

Dictionary-based SHM methods sidestep the computational complexity of TOF methods by constructing a matrix of signal profiles, where each column corresponds to the impulse response of the structure with a crack at a certain location on a predefined grid [33, 34]. The locations of a small number of cracks in the structure are then determined by solving a sparse recovery problem, where the support of the sparse signal vector $x$ in equation (1) corresponds to the damage locations [34]. In dictionary-based SHM methods, each row of the sensing matrix $\Phi$ corresponds to a measured time instance of a reflected signals (see Fig. 4). Applying a sensor selection algorithms to the SHM matrix will reduce the number of measurements required while preserving the damage localization performance.

To compare Insense against the other baseline algorithms, we use real SHM data from a crack localization experiment on a metal plate with 25 potential cracks located on a $5\times 5$ square grid. The experimentally measured signals of size $D=2250$ from all cracks are obtained and stored in a matrix $\Phi\in\mathbb{R}^{2250\times 25}$. The crack locations are then identified by solving the sensor selection problem with the experimentally measured sensing matrix $\Phi$. Fig. 4 visualizes the $M=10$ selected time instances for Insense and two other baseline algorithms, and Table IV showcases their performance on BP recovery in localizing $K=2$ cracks using $M\in\{5,6,7,8,9,10\}$ time instances. MI-G and Determinant-G does not run on our desktop PC with 12 GB of RAM due to high memory requirements. Insense ran faster than the optimization-based algorithm Convex SS and only slightly slower than the greedy algorithms.

In summary, in this SHM application, Insense consistently outperforms the baseline algorithms in terms of crack localization accuracy and reduces the number of time measurement instances required for accurate localization. We note that the measurements are not statistically independent in this dataset, since they correspond to measurements of the same reflected signal at different time instances, highlighting the superior performance of Insense in applications where the independence assumption in the sensing model is slightly violated.

VI Conclusions

In this paper, we developed the Incoherent Sensor Selection (Insense) algorithm for the underdetermined sensor selection problem that optimizes the average squared coherence of the columns of the selected sensors (rows) via a computationally efficient relaxation. Our synthetic and real-world data results have both verified the utility of the average squared coherence metric and the performance of the Insense algorithm. In particular, Insense provides superior performance than existing state-of-the-art sensor selection algorithms, especially in the real-world problems of microbial diagnostics and structural health monitoring.

There are a number of avenues for future work. One is finding computational shortcuts for operating on large-scale sensing matrices. Another is exploring different variants of the Insense algorithm that select sensors under generalized sparse models, e.g., group-sparse, tree-sparse and low-rank models.

Acknowledgment

We thank S. Nagarajaiah for providing the Structural Health Monitoring data. A. Aghazadeh, A. S. Lan, and R. G. Baraniuk were supported by National Science Foundation (NSF) grants CCF-1527501 and CCF-1502875, Defense Advanced Research Projects Agency (DARPA) Revolutionary Enhancement of Visibility by Exploiting Active Light-fields grant HR0011-16-C-0028, and Office of Naval Research (ONR) grant N00014-15-1-2735. M. Golbabaee was supported by the Swiss NSF research grant PBELP2-146721.