Improved Hyperspectral Unmixing With Endmember Variability Parametrized Using an Interpolated Scaling Tensor
Ricardo Augusto Borsoi, Tales Imbiriba, Jos\'e Carlos Moreira Bermudez

TL;DR
This paper introduces a two-step hyperspectral unmixing method that leverages pure pixel information and low-rank spectral variability modeling to improve unmixing accuracy and robustness.
Contribution
It proposes a novel approach that separates the estimation of spectral variability from abundance calculation, enhancing stability and exploiting pure pixel information effectively.
Findings
Improved unmixing accuracy on synthetic and real data.
Effective modeling of spectral variability as a smooth, low-rank tensor.
Enhanced robustness against ill-posed optimization problems.
Abstract
Endmember (EM) variability has an important impact on the performance of hyperspectral image (HI) analysis algorithms. Recently, extended linear mixing models have been proposed to account for EM variability in the spectral unmixing (SU) problem. The direct use of these models has led to severely ill-posed optimization problems. Different regularization strategies have been considered to deal with this issue, but none so far has consistently exploited the information provided by the existence of multiple pure pixels often present in HIs. In this work, we propose to break the SU problem into a sequence of two problems. First, we use pure pixel information to estimate an interpolated tensor of scaling factors representing spectral variability. This is done by considering the spectral variability to be a smooth function over the HI and confining the energy of the scaling tensor to a…
| Synthetic HI | Houston HI | ||||
|---|---|---|---|---|---|
| FCLS | 0.0416 | – | – | 0.0212 | 0.0478 |
| PLMM | 0.0353 | 0.0220 | 0.0232 | 0.0117 | 0.0360 |
| ELMM | 0.0373 | 0.0135 | 0.0174 | 0.0121 | 0.0033 |
| GLMM | 0.0343 | 0.0128 | 0.0163 | 0.0118 | 0.0003 |
| Proposed | 0.0233 | 0.0130 | 0.0164 | 0.0123 | 0.0085 |
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
IMPROVED HYPERSPECTRAL UNMIXING WITH ENDMEMBER VARIABILITY PARAMETRIZED USING AN INTERPOLATED SCALING TENSOR
Abstract
Endmember (EM) variability has an important impact on the performance of hyperspectral image (HI) analysis algorithms. Recently, extended linear mixing models have been proposed to account for EM variability in the spectral unmixing (SU) problem. The direct use of these models has led to severely ill-posed optimization problems. Different regularization strategies have been considered to deal with this issue, but none so far has consistently exploited the information provided by the existence of multiple pure pixels often present in HIs. In this work, we propose to break the SU problem into a sequence of two problems. First, we use pure pixel information to estimate an interpolated tensor of scaling factors representing spectral variability. This is done by considering the spectral variability to be a smooth function over the HI and confining the energy of the scaling tensor to a low-rank structure. Afterwards, we solve a matrix-factorization problem to estimate the fractional abundances using the variability scaling factors estimated in the previous step, what leads to a significantly more well-posed problem. Simulations with synthetic and real data attest the effectiveness of the proposed strategy.
**Index Terms— ** Hyperspectral data, endmember variability, GLMM, pure pixel, tensor interpolation.
1 Introduction
Spectral Unmixing (SU) is part of a number of algorithms that retrieve vital information from hyperspectral images (HIs) in many applications [1]. SU aims at extracting the spectral signatures of materials present in the HI of a scene, as well as the proportion in which they contribute to each HI pixel. Many parametric models have been proposed to describe the interaction between light and the target surface [1, 2]. The simplest of such models is the Linear Mixing Model (LMM), which considers that the observed reflectance of an HI pixel is obtained from a convex combination of the spectral signatures of pure materials. This model imposes a convex geometry to the SU problem, where all HI pixels are confined to a simplex whose vertices are the reflectances of pure materials, usually termed endmembers (EMs). The linearity and convexity of the LMM model lead to an interpretation of its coefficients as the relative abundances of each pure material in the HI. Nevertheless, some characteristics of practical HIs cannot be modeled by the standard LMM, such as nonlinearities [2, 3, 4, 5] or variations of the EMs along the image [6, 7, 8]. More sophisticated models are required when such nonidealities have important impact on the HI.
EM variability can result, for instance, from environmental conditions, illumination, atmospheric or temporal changes [9]. Its occurrence may incur the propagation of significant estimation errors throughout the unmixing process [6]. Most of the methods proposed so far to deal with spectral variability can be classified in three major groups: endmembers as sets, endmembers as statistical distributions and, more recently, methods that incorporate the variability in the mixing model, often using physically motivated concepts [10]. The method proposed in this work combines elements from the first and third groups. Specifically, we adopt a modified mixing model over a very specific variational set.
In the first group, EM variability is addressed by considering different types of variational sets, among which spectral bundles are prominent [9]. One nice property of these methods is that the bundles can be directly extracted from the observed HI. Few works expand this idea by adopting pixel-dependent EMs obtained using some kind of spatial interpolation of the original EM signatures [11, 12, 13]. These interpolated EMs sets are then used to unmix the data using the standard LMM. This type of approach often profits from a priori information about EM spectra (pure pixels) and their position in the image. Interpolation strategies include linear regression [11, 14, 15] and kriging [12, 13]. However, these strategies lack flexibility to adapt to variations in the endmember signatures that lie beyond the descriptive capability of the spectral bundles.
In the third group, different extensions of the LMM have been proposed to cope with spectral variability [6, 7, 8]. The models differ with respect to the type of variability they represent and their physical motivation. The Perturbed LMM model (PLMM) [6] introduces an additive perturbation to the EM matrix. It allows the modeling of arbitrary spectral variations, but lacks physical motivation. The Extended LMM (ELMM) [7] and its generalization, the Generalized LMM (GLMM) [8], are physically motivated, but differ in their ability to model arbitrary variability. The ELMM performs well if spectral variability is due mainly to illumination variations, but it lacks flexibility when the EMs are subject to more complex spectral distortions. The GLMM is able to model arbitrary EM variability by considering band-dependent multiplicative factors, thus linking the amount of spectral variability to the magnitude of the EM reflectance in each band. This effect is consistent with empirical observations, leading to a more physically reasonable model. One drawback of extended LMMs is that they lead to ill-posed optimization problems. Hence, regularization strategies must be devised to yield meaningful results [16, 17]. Different regularization strategies have been recently proposed. For instance, [18] and [19] proposed a data-dependent multiscale strategy to exploit the spatial uniformity existing in HIs. The approaches in [20, 21] assume that most of the energy of HIs is confined to a low-dimensional structure within the image space, and exploit low-rank tensor decomposition techniques to impose regularity to the solution.
In this work we propose a hybrid approach by leveraging pure pixel information in the context of parametric endmember models. Specifically, we propose to break the SU problem into two sequential problems. We first use pure pixel information to estimate an interpolated tensor of spectral variability scaling factors by assuming that its energy is confined to a low-rank structure. Afterwards, we solve a matrix-factorization problem to estimate the fractional abundances. This is done by using the variability scaling factors estimated in the previous step in the form of a regularization term, which leads to a significantly more well-posed problem. We adopt the GLMM to parametrically model spectral variability, as it combines flexibility and physical motivation.
This paper is organized as follows. Section 2 briefly reviews the LMM and its GLMM extended version. Section 3 introduces the proposed formulation and the corresponding SU algorithm. The performance of the proposed method is compared with competing algorithms in Section 4. Finally, the conclusions are presented in Section 5.
2 Linear Mixing Models
The Linear Mixing Model (LMM) [1] assumes that a given a pixel , with bands, is represented as
[TABLE]
where is a matrix whose columns are the endmember spectral signatures , is the abundance vector, is an additive white Gaussian noise (WGN) and is the entrywise operator. The LMM assumes that the endmember spectra are fixed for all pixels , , in the HI. This assumption jeopardizes the accuracy of estimated abundances in many circumstances due to the spectral variability existing in a typical scene.
To mitigate this issue, the GLMM [8] models arbitrary EM variability by considering band-dependent multiplicative factors, which links the amount of spectral variability to the magnitude of the EM reflectance in each band.
The GLMM introduces a scaling matrix with entries , which acts individually on each wavelength. The model represents the -th HI pixel as
[TABLE]
where stands for the Hadamard product.
3 Proposed Unmixing Algorithm
Given a parametric extended linear mixing model , and an HI , the SU problem can be generally cast as the determination of and that minimize a risk functional of the form
[TABLE]
where is a 3-D tensor obtained by stacking all pixel-dependent endmember matrices , such that , is the abundance matrix, and and are regularization terms to improve the problem conditioning. Although parametric EM models such as the ELMM and GLMM have enough flexibility to adequately represent the variability in the endmember spectra, the resulting unmixing problem is highly ill-posed. This problem is accentuated by the fact that previous formulations such as [16, 8] only considered a single reference, or average endmember matrix, as a priori information (or initialization), while estimating the variability maps and endmembers for all pixels in the scene. Although good performance has been reported in [16, 8], the correct estimation of the parametric models for complex scenes remains a challenge.
A characteristic of many scenes, which remains unexplored in this context, is the existence of multiple pure pixels in an observed HI. This information can be leveraged to help in the estimation of the parametric endmember model, introducing more information into the problem and reducing its ill-posedness.
We propose to break the unmixing problem into a sequence of two problems:
- (i)
Using pure pixel information extracted from the HI by standard EM extraction algorithms and assuming smooth spectral variability throughout the HI, estimate an interpolated tensor of variability factors whose energy is assumed to be confined to a low-rank structure.
- (ii)
Using the estimated tensor as a prior information in a standard matrix-factorization problem, solve it to estimate the endmember and abundance matrices.
3.1 Estimating the Tensor of Variability Factors
The objective of this first problem is to estimate the scaling matrices , , in (2). To this end, we assume:
- a)
the availability of a reference endmember matrix , which can be obtained using any endmember extraction method;
- b)
the knowledge of the set of locations of pure pixels for the -th endmember, for all ; 111Pure pixels are here defined here as a set of pixels whose spectral distance relative to the reference EMs in is less than a specified threshold.
- c)
that the EM variability changes smoothly in the HI, so that each endmember variability function has most of its energy confined to a low-dimensional structure in the image space.
Consider the 4-D tensor , , of variability weights with entries , corresponding to the scaling factor to be applied to the -th band of the -th endmember in the pixel at position of the HI. generalizes the GLMM scaling factors. We propose to estimate the variability tensor by solving the following optimization problem:
[TABLE]
In (4), is a low-rank tensor which imposes a low-rank constraint on the variability maps to enforce the smooth variation. is an tensor of ones, so that is a regularization term that enforces the variability to be small, and whose importance is controlled by the parameter . represents the pixel at position , is the -th column of with elements , and the third term enforces the similarity between pure pixels at known positions and their corresponding endmembers and variability maps. Its strictness is controlled by . The associated optimization problem is given by
[TABLE]
This problem is non-convex and generally NP-hard due to the rank constraint. We propose to solve it using the alternating optimization approach described in Algorithm 1, as follows.
3.1.1 Optimizing with respect to
To solve problem (5) with respect to we consider fixed. Thus, the optimization problem with respect to is given by
[TABLE]
By taking the derivative with respect to each position of and setting it equal to zero, we end up with the following solution:
[TABLE]
for , and , .
3.1.2 Optimizing with respect to
The optimization problem with respect to is given by
[TABLE]
where we write tensor as
[TABLE]
where denotes the outer product and is a small number since we assume that most of the energy of lies in a low-rank structure. Using (9) in (8) leads to the following optimization problem:
[TABLE]
where {\boldsymbol{\Xi}}=\text{Diag}_{4}\big{(}\xi_{1},\ldots,\xi_{r}\big{)} is an order-4 diagonal tensor with . Problem (10) can be solved using an alternating least-squares strategy [22]. The solution is then obtained using the full multilinear product [21]:
[TABLE]
3.2 The Spectral Unmixing Problem
Given a reference EM matrix and the EM scaling factors for each pixel (with each corresponding to a single pair ), the SU problem can be formulated as the minimization of the risk functional (3) with respect to and , using an appropriate EM model and regularization terms. We propose to minimize the following regularized cost functional:
[TABLE]
The last term in (3.2) promotes spatial regularity in the abundance maps. and are linear operators that compute the horizontal and vertical gradients of a bidimensional signal, acting separately for each material of , and is the norm.
Cost function in (3.2) is non-smooth and non-convex with respect to both and , but convex with respect to each of them. Following the approaches used in [7, 8], we search for a locally optimal solution by alternately minimizing (3.2) with respect to each variable, as described in Algorithm 2.
3.2.1 Optimization with respect to
Considering only the terms in (3.2) that depend on , the optimization problem for the EM tensor becomes
[TABLE]
Problem (13) can be solved individually for each pixel . Relaxing the nonnegativity constraint on , the solution is given by
[TABLE]
An approximate solution to the original constrained problem (13) can then be obtained by projecting onto the nonnegative orthant by attributing zero to any negative entries [7, 8].
3.2.2 Optimization with respect to
Considering only the terms in (3.2) that depend on , the optimization problem for the abundance matrix becomes
[TABLE]
This problem is non-smooth and not separable with respect to the different pixels in the HI. Nevertheless, problem (3.2.2) can be solved efficiently using the Alternating Direction Method of the Multipliers (ADMM) [23]. The procedure is described in detail in [7].
4 Experimental Results
We now illustrate the performance of the proposed method through simulations with both synthetic and real data. We compare the proposed method with the fully constrained least squares (FCLS), the ELMM [7], the PLMM [6] and the GLMM [8]. In all experiments, the reference EM matrix was extracted from the observed HI using the VCA algorithm [24]. The sets of pure pixel locations were estimated by selecting the pixels with the smallest spectral angle relative to the reference EMs in . The performances were evaluated using the Root Means Squared Error (RMSE) between the estimated abundance maps (), between the EM matrices () and between the reconstructed images (). The RMSE between two generic tensors is defined as
[TABLE]
where denotes the number of elements in the tensor .
We consider also the Spectral Angle Mapper (SAM) to evaluate the estimated EM tensor
[TABLE]
4.1 Synthetic data
A synthetic dataset was created from spatially correlated abundance maps containing a significant amount of pure pixels. The HI was constructed using reference EMs extracted from the USGS Spectral Library [25]. Spectral variability was introduced following the GLMM model in Eq. (2) to generate pixel-dependent EM signatures with spatial and spectral correlation imposed on using a 3-D Gaussian filter. Then, WGN was added to yield a 30dB SNR.
We selected the optimal parameters for each algorithm by performing a grid search using the ranges of parameters suggested by the authors in the original publications. For PLMM we used and searched for and in the range and , respectively. For ELMM, GLMM and the proposed method, we selected the parameters in the following ranges: , , and . We also used and fixed in Algorithm 1. Finally, the number of pure pixels extracted from the image was , and for the first, second and third EMs, respectively.
The results are shown in Table 1. It can be seen that the proposed method yields a significantly smaller than the other algorithms. It also yields the second-best results and , both very close to the best results obtained by the GLMM. The reconstruction errors are smaller for PLMM, GLMM and ELMM, which is expected since these methods have a larger number of degrees of freedom, while the proposed method estimates the variability scaling factors a priori. However, smaller reconstruction errors do not necessarily imply better abundance estimation, as evidenced by the results. The execution time of the proposed method is , a modest increase in complexity when compared to the benefits in terms of estimation accuracy.
4.2 Real data
For simulations with real data we considered the Houston dataset, which is known to have four EMs [7, 8]. The parameters for the proposed method were set as , , , and . For the other algorithms we used the same parameters [8]. The sets were constructed by extracting all pixels in the HI with an angle smaller than to the reference EMs. The reconstructed abundance maps for all tested algorithms are shown in Fig. 1, while the reconstruction errors are presented in Table 1.
It can be seen that the proposed method provides abundance estimation that is accurate and comparable with the results obtained using ELMM or GLMM. Furthermore, the proposed method results in stronger concrete and vegetation components in the stadium stands and in the field, respectively, when compared to the other algorithms. As in the synthetic data example, the results in Table 1 show smaller values for GLMM and ELMM. Again, this is not directly related to better abundance estimation. The proposed method demanded a computational time equal to .
5 Conclusions
We have proposed a new spectral unmixing algorithm accounting for spectral variability. Modeling the spectral variability as a smooth function over the image, pure pixel information was leveraged from the HI to estimate a tensor of EM scaling factors prior to unmixing. The use of this predetermined tensor has led to a much more well-posed SU problem when compared to current algorithms, in which the scaling factors are blindly estimated from the image. Simulations using synthetic and real data indicate that the proposed method can lead to significant improvements in abundance estimation accuracy.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] N. Keshava and J. F. Mustard, “Spectral unmixing,” IEEE Signal Processing Magazine , vol. 19, no. 1, pp. 44–57, 2002.
- 2[2] N. Dobigeon, J.-Y. Tourneret, C. Richard, J. C. M. Bermudez, S. Mc Laughlin, and A. O. Hero, “Nonlinear unmixing of hyperspectral images: Models and algorithms,” IEEE Signal Processing Magazine , vol. 31, no. 1, pp. 82–94, Jan 2014.
- 3[3] T. Imbiriba, J. C. M. Bermudez, C. Richard, and J.-Y. Tourneret, “Nonparametric detection of nonlinearly mixed pixels and endmember estimation in hyperspectral images,” IEEE Transactions on Image Processing , vol. 25, no. 3, pp. 1136–1151, March 2016.
- 4[4] T. Imbiriba, J. C. M. Bermudez, and C. Richard, “Band selection for nonlinear unmixing of hyperspectral images as a maximal clique problem,” IEEE Transactions on Image Processing , vol. 26, no. 5, pp. 2179–2191, May 2017.
- 5[5] T. Imbiriba, J. C. M. Bermudez, J.-Y. Tourneret, and C. Richard, “Detection of nonlinear mixtures using gaussian processes: Application to hyperspectral imaging,” in ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing , Jan 2014, pp. 7949–7953.
- 6[6] P.-A. Thouvenin, N. Dobigeon, and J.-Y. Tourneret, “Hyperspectral unmixing with spectral variability using a perturbed linear mixing model,” IEEE Trans. Signal Processing , vol. 64, no. 2, pp. 525–538, Feb. 2016.
- 7[7] L. Drumetz, M.-A. Veganzones, S. Henrot, R. Phlypo, J. Chanussot, and C. Jutten, “Blind hyperspectral unmixing using an extended linear mixing model to address spectral variability,” IEEE Transactions on Image Processing , vol. 25, no. 8, pp. 3890–3905, 2016.
- 8[8] T. Imbiriba, R. A. Borsoi, and J. C. M. Bermudez, “Generalized linear mixing model accounting for endmember variability,” in Acoustics, Speech and Signal Processing (ICASSP), 2018 IEEE International Conference on . IEEE, 2018, pp. 1862–1866.
