More chemical detection through less sampling: amplifying chemical signals in hyperspectral data cubes through compressive sensing

TL;DR

This paper demonstrates that compressive sensing sampling of hyperspectral data cubes can unexpectedly amplify chemical signals, often making them more detectable than in fully sampled data, with potential practical benefits.

Contribution

It reveals a novel phenomenon where bandwise compressive sensing enhances chemical signal detection in hyperspectral data, supported by real-world dataset analysis and theoretical discussion.

Findings

Chemical signals can be amplified through CS sampling and reconstruction.

Amplification of signals increases as sampling becomes more sparse.

CS-based reconstruction can outperform full sampling in chemical detection.

Abstract

Compressive sensing (CS) is a method of sampling which permits some classes of signals to be reconstructed with high accuracy even when they were under-sampled. In this paper we explore a phenomenon in which bandwise CS sampling of a hyperspectral data cube followed by reconstruction can actually result in amplification of chemical signals contained in the cube. Perhaps most surprisingly, chemical signal amplification generally seems to increase as the level of sampling decreases. In some examples, the chemical signal is significantly stronger in a data cube reconstructed from 10% CS sampling than it is in the raw, 100% sampled data cube. We explore this phenomenon in two real-world datasets including the Physical Sciences Inc. Fabry-P\'{e}rot interferometer sensor multispectral dataset and the Johns Hopkins Applied Physics Lab FTIR-based longwave infrared sensor hyperspectral dataset. Each of these datasets contains the release of a chemical simulant, such as glacial acetic acid, triethyl phospate, and sulfur hexafluoride, and in all cases we use the adaptive coherence estimator (ACE) to detect a target signal in the hyperspectral data cube. We end the paper by suggesting some theoretical justifications for why chemical signals would be amplified in CS sampled and reconstructed hyperspectral data cubes and discuss some practical implications.

Figures

Equations

$$y=S\tilde{x}\in {\mathbb{R}}^k$$

$$u^{\ast }=\underset{u\in {\mathbb{R}}^n}{\mathrm{argmin}}||u|{|}_{{\ell }_1}\text{such that }y=S{H}^{-1}u,.$$

$$U^{\ast }=\underset{U\in {\mathbb{R}}^{n\times b}}{\mathrm{argmin}}||U|{|}_{{\ell }_1}\text{such that }Y=S{H}^{-1}U,$$

$$\frac{(s^T{\Gamma }^{-1}x{)}^2}{(s^T{\Gamma }^{-1}s)(x^T{\Gamma }^{-1}x)},$$

$$1-\prod _{i=1}^M(1-c_i).$$

Full text

$\ell_{1}$-regularization.

1 INTRODUCTION

One of the most important applications of hyperspectral imaging is to the problem of detecting specific chemicals in a given scene. Unfortunately, hyperspectral images are generally much larger than traditional images even when one is collecting relatively few bands, and hyperspectral devices also generally come with a high price tag. For these reasons, any methods that allow hyperspectral images to be sampled at lower rates while at the same time retaining their discriminative ability have many applications. In particular, strategies for reduction of the number of sensors are quite valuable. Compressive sensing (CS) is exactly such a framework. In particular, CS provides methods for accurately reconstructing under-sampled signals.

Applying CS to hyperspectral imaging is an active area of research, with a multitude of different approaches. These include CS sampling schemes that are performed to different extents in both the spatial and the spectral domain. Several works have gone beyond simply trying to reconstruct the image itself and instead make the process of signal unmixing, i.e. identifying the different spectral signals in an image, a component of the reconstruction process [1, 2, 3]. Other works have explored reconstruction frameworks specifically designed to utilize the structure of hyperspectral data [4]. Finally, various CS strategies have been proposed for data extracted from devices specific to hyperspectral imaging [5].

In this paper we explore chemical detection in CS reconstructions of hyperspectral images after low levels of sampling. We show that, surprisingly, not only can a chemical still be detected in a hyperspectral image that has been reconstructed after 10% sampling, but at least in some examples, the chemical signature can be slightly stronger in the reconstructed image. This is a surprising result because naively, one would expect that by using 10% sampling, one is losing 90% of the information from the hyperspectral image. We note that signal amplification only seems to happen with low sampling levels. An interesting future direction would be to understand in which situations precisely a signal is amplified by reconstruction.

This paper is organized as follows. In Sec. 2.1, we give a brief summary of some of the ideas underlying CS. In Sec. 2.2, we set up some of the mathematical framework and notation for discussing hyperspectral imagery. In Sec. 2.3, we describe the chemical detection algorithm (ACE) used in experiments for this paper. The main results of the paper are discussed in Sec. 2.4. Finally, in Sec. 2.5, we suggest some possible explanations for the chemical signal amplification seen in reconstructed images. In Sec. 3 we suggest some directions for future research.

2 BACKGROUND

In this paper we will generally use upper case letters (for example $S$, $X$, $U$, $H$, and $Y$) to denote matrices. We will use lower case letters (for example $u$, $u^{*}$, $x^{\prime}$, $\tilde{x}$, and $y$) to denote vectors. Matrices being placed side by side always denotes standard matrix multiplication.

2.1 Compressive sensing and the single-pixel camera framework

Compressive sensing (CS) is a collection of methods that permit highly accurate reconstruction of certain classes of signals even when they have been sampled well under Nyquist-rate [6]. When it is applicable, CS allows one to solve the ill-posed problem of finding $\tilde{x}$ from

[TABLE]

when $S$ is a $k\times n$ matrix, $\tilde{x}\in\mathbb{R}^{n}$, and $k<n$. A basic requirement for application of methods from CS for solution of (1) is that we can make some assumption about the signal $\tilde{x}$. This allows us to choose $\tilde{x}$ (or a close approximation to $\tilde{x}$) out of the infinite number of solutions $x^{\prime}$ to $Sx^{\prime}=y$. One common choice of assumption about $\tilde{x}$ that is used frequently in CS is that it will be sparse in some particular basis. Although when $\tilde{x}$ is an image it will almost never be sparse in its natural basis, it is generally true that it will be compressible (that is, approximately sparse) in a wavelet basis.

Let $H,H^{-1}:\mathbb{R}^{n}\rightarrow\mathbb{R}^{n}$ be the $1$-dimensional Haar wavelet transformation and its inverse respectively. Then the CS reformulation of (1) is to solve:

[TABLE]

In words, this optimization problem seeks $x$ satisfying $Sx=y$ such that $x$ is maximally sparse in the $1$-dimensional Haar wavelet basis, that is, $||u||_{\ell_{1}}$ is minimized for $u=Hx$.

It is important to note that even though the solution $x^{*}=H^{-1}u^{*}$ to (2) may often be a very good approximation to $\tilde{x}$, we always expect some information to be lost in the process of sampling $\tilde{x}$ with $S,$ in particular when $k\ll n$. In some cases the information lost in this process is mostly noise, leading to a reconstruction $x^{*}$ that is preferable to the original signal $\tilde{x}$. For example, another variant of CS reconstruction which minimizes total variation [7] is closely related to a family of denoising algorithms called total variation denoising. Such an observation may form the first step in explaining the phenomenon described in this paper.

2.2 Framework and notation for the compressive sensing of hyperspectral data

Hyperspectral data is highly structured and it is essential to capture this structure when performing optimization routines such as (2). In this paper we assume that a single band of our hyperspectral data has size $n$ (that is, as a 2-dimensional array it has size $n_{1}\times n_{2},$ where $n=n_{1}n_{2}$) and that there are $b$ bands. Given the above conventions, we will realize an $n_{1}\times n_{2}\times b$ hyperspectral data cube as an $n\times b$ matrix $X\in\mathbb{R}^{n\times b}$, where we have flattened each band to a column vector in $X$. Then, $U=HX$ is an $n\times b$ matrix where the columns of $U$ are the columns of $X$ transformed into the $1$-dimensional Haar wavelet basis. Our optimization problem (2) applied to a hyperspectral image is thus

[TABLE]

where $S$ is again a sampling matrix in $\mathbb{R}^{k\times n}.$ The $\ell_{1}$-norm above is applied to $U$ in the same way as it would be to $U$ flattened to a length $nb$ vector. Note that the expression $H^{-1}U$ is equivalent to taking the inverse of the 1-dimensional Haar wavelet transform for every band in the data cube flattened to a vector. We call the $k\times b$ output $Y=SX$ a $(100\frac{k}{n})$% sampling of $X$. In this paper we will generally sample at $10\%$, so that we are effectively throwing out $90\%$ of the information in a data cube. It is for this reason that it is surprising that chemical signals in $X$ sometimes become stronger.

There has been considerable research toward developing feasible optimization problems that reconstruct the bands of a hyperspectral image non-independently [4]. Such methods make use of the correlation between bands. We chose to study (3) because in many cases hardware and sampling constraints force bands to be reconstructed independently. It would be interesting to understand if solving more hyperspectral specific optimization problems also produce reconstructions with signal enhancement.

In all the experiments described in this paper, we constructed $S$ via a modified Walsh-Hadamard matrix [8]. We suggest investigation into whether other sampling methods also result in signal enhancement at low sampling levels.

There are a large number of algorithms that have been developed for solving optimization problems such as (2) and (3). We choose to utilize the split Bregman method [9] since it is fast, lightweight, and gives reliable convergence.

2.3 Detection algorithms

Since the point of our experiments was to demonstrate a phenomenon in which compressive sensing followed by reconstruction of a hyperspectral image amplifies chemical signals, choosing the appropriate chemical detection algorithm was a key component of our work.

In our experiments we use the adaptive coherence/cosine estimator (ACE), a well-known technique used for chemical detection [10, 11]. Let $s$ be a target spectral signature and let $x$ be a spectral signature in a specific pixel within a hyperspectral cube (in the literature, $x$ is the pixel under test (PUT)). The ACE statistic is the square of the cosine of the angle between $s$ and $x$ relative to the background. To be precise, the ACE statistic is calculated as

[TABLE]

where $\Gamma$ is the maximum likelihood estimator for the covariance matrix of background data.

In addition to using ACE for chemical detection, we compute the bulk coherence (multipulse coherence) estimator [12, 13, 14]. The bulk coherence statistic enhances the signal in neighborhoods that have several pixels with relatively high ACE values, a property that is appropriate for chemical release settings. If $c_{i}$ is the ACE statistic for pixel $i$ and a neighborhood of pixels is indexed by $i=1,\ldots,M,$ then the bulk coherence statistic is

[TABLE]

ACE values near one result in a product whose terms are close to zero, resulting in a high bulk coherence value (near one). Experimentally, the bulk coherence statistic leads to improved chemical detection. We also add one additional filter that we refer to as persistence in some cases: we set the value associated to a pixel to zero if its bulk coherence value doesn’t stay above a pre-specified threshold for at least five consecutive time steps.

To facilitate objective comparison, we further incorporate an algorithmic definition of a threshold for the ACE statistic (similarly for bulk coherence) [15]. We can then declare a chemical to be present or absent in a given pixel based on comparison of the ACE statistic against this threshold. The threshold definition is motivated by the idea that the threshold should be slightly larger than the ACE values observed in typical background cubes (cubes that have been collected with the intended device in which it is known that the target chemical is absent). The algorithmic determination of the threshold is responsive to the device used to sense the data, the reconstruction method, and the spectral signature of the target chemical.

2.4 Signal enhancement in chemical detection

We demonstrate the signal enhancement phenomenon on two hyperspectral datasets. The first is the Fabry-Pérot interferometer sensor multispectral dataset[16] and the second is the Johns Hopkins Applied Physics Lab FTIR-based longwave infrared sensor hyperspectral dataset [17]. In our analysis of each data set, we restrict to a $64\times 64$ spatial field of view in which the chemical release is observed. Figures 1-3 relate to the SF6 27 Romeo release video from the Johns Hopkins dataset. This video consists of $140$ hyperspectral images. Between image $20$ and $30,$ sulfur hexafluoride (SF6) gas is released and disperses. This release is apparent in Fig. 1, which shows the number of pixels that the ACE algorithm indicates contain the chemical as a function of time (the number of such pixels in the raw data is indicated by the dashed line and the number of pixels in the data reconstructed from $10\%$ sampling is indicated by the solid black line). It can be seen that during the release itself, more pixels in the reconstructed data have ACE values above the threshold than do those in the raw data. The chemical returns to the scene in a dissipated form sometime around cube 70 (presumably due to a change in wind direction). In this case, the uncompressed data results in slightly stronger chemical detection, with a few exceptions for short time frames.

The specific distributions of bulk coherence ACE values with persistence for hyperspectral image $30$ (both raw (left) and reconstructed (right)), is shown in Fig. 2. As this histogram indicates, the spread between values corresponding to spatial locations containing the chemical signature and those that do not is much larger in the reconstructed image. This is useful as it makes distinguishing pixels that do and do not contain the chemical easier. For reference the threshold value for chemical presence in the raw image is $0.0077$ and the threshold value for chemical presence in the reconstructed image is $0.2656$. A similar phenomenon is seen in Fig. 3 but for cube $90$ where most (but not all) the chemical has already dispersed from the scene.

We next examine two hyperspectral videos in the Fabry-Pérot interferometer sensor multispectral dataset. The first contains a release of the chemical methyl salicylate (MeS). This release is shown in Fig. 4. As before the $x$-axis is the frame in the hyperspectral video and the solid black and dashed blue curve give the number of pixels that ACE indicates contain the chemical for reconstructed and uncompressed data, respectively. As can be seen, this is noisy data with many false positives (which appear as spikes). Despite the fact that the chemical signature in this dataset is quite weak even at the peak of the release, the signal has approximately the same strength in the reconstructed image compared to the raw image.

Figure 5 gives a histogram of ACE bulk coherence values for the MeS release at image 80 when the chemical is present in the scene. Unlike what was seen in Figs. 2 and 3, there is no longer any clear separation between spatial locations containing the signature for MeS and spatial locations that do not. The fact that there is greater spread of values in the reconstructed image however makes classification of regions containing the chemical easier and more stable.

Finally, the Fabry-Pérot interferometer sensor multispectral dataset also contains a hyperspectral video of release of triethyl phosphate (TEP). A plot of this chemical release as captured by ACE is shown in Fig. 6. This is an example where results of detection in the raw and reconstructed images are mixed. While the signal is better detected in raw data up to the peak of the release, it appears to be the case that the reconstructed data results in better detection in later frames as the chemical dissipates. This illustrates the point that signal amplification is not always consistent. Examples exist in which the signal is either slightly weaker or of equal strength in reconstructed images. Another important observation (which is discussed in Sec. 2.5) is that the false positives that appear in raw images are effectively eliminated in reconstructed images, pointing toward the reconstruction algorithm functioning as a denoising algorithm.

To give the reader a sense of what these improvements look like in practice, in Fig. 7 we show three pairs of frames of ACE detection for both raw data (on the left) and data reconstructed from just $10\%$ CS sampling (on the right). One can see that there are certain instances (such as (a)-(b)) where CS sampling highlights a signal that otherwise would not be noticeable. In (c)-(d), we see that the chemical signal has been amplified but there is a question about whether it is as spatially accurate. Finally, in (e)-(f), CS sampling and reconstruction simply strengthens a signal that is already present, likely showing spatial regions where the signal was too weak to have been seen before.

2.5 Possible theoretical explanation and analysis

There are a number of possible explanations for the signal amplification phenomenon observed in hyperspectral data reconstruction by solving (2). The first is that sampling $\tilde{U}$ and then using this sample to reconstruct the approximation $U$ of $\tilde{U}$ is effectively denoising $\tilde{U}$. Indeed, as was noted in Fig. 6, the rate of false positives of TEP detection is strongly reduced in the reconstructed data compared to the raw data. On the other hand, the denoising effects are not supported by Fig. 4 where high rates of false positives appear in both the raw and reconstructed data.

Another facet worth further investigation is the apparent trade-off between spatial accuracy and signal detection. As can be seen in Fig. 7, while signal strength is stronger in reconstructed cubes, the spatial accuracy may sometimes be reduced. This would be consistent with the sampling strategy used in this paper where sampling was done in the spatial domain but not the spectral domain. If, more generally, there is a tradeoff between spatial and spectral accuracy, then the phenomenon of signal amplification in reconstructed data will be more applicable to situations where detecting the presence of a chemical is valued above understanding precise spatial locations of chemicals within the field of view.

3 Conclusion

In this paper we described a phenomenon in which hyperspectral images sampled at very low levels and reconstructed using techniques from CS not only contain strong chemical signals, but sometimes even contain amplified signals. This observation suggests some new directions for research, the most important of which is to explain why this is happening. While we suggest some possible explanations in Section 2.5, much more work needs to be done in this direction. Some other questions that should be investigated include:

• •

Does this phenomenon occur when a different reconstruction framework (or optimization problem) is used? It would be especially interesting to know whether this occurs in some CS frameworks specifically designed for hyperspectral imaging (e.g. [4]).

• •

Would similar results be obtained when using a different sparsifying basis (i.e. instead of the Haar wavelet basis)?

• •

In the experiments described in this paper, the same modified Walsh-Hadamard sampling basis was consistently used [8]. Does signal amplification still occur when different sampling strategies are used?

Acknowledgements.

The authors would like to thank Louis Scharf for insightful discussions related to this work, especially with regard to content involving ACE and MPACE. This research was partially supported by Department of Defense Army STTR Compressive Sensing Flash IR 3D Imager contract W911NF-16-C-0107 and Department of Energy STTR Compressive Spectral Video in the LWIR contract W911SR-17-C-0012.