Superpixel Contracted Graph-Based Learning for Hyperspectral Image Classification

TL;DR

This paper introduces a novel superpixel-based graph framework for hyperspectral image classification that achieves high accuracy with limited labeled data, outperforming existing methods.

Contribution

It presents a new superpixel method for hyperspectral data and a graph-based semi-supervised classifier that improves classification accuracy with minimal labeled samples.

Findings

Effective in small labeled data scenarios

Outperforms several state-of-the-art techniques

Speeds up graph-based classifiers for hyperspectral images

Abstract

A central problem in hyperspectral image classification is obtaining high classification accuracy when using a limited amount of labelled data. In this paper we present a novel graph-based framework, which aims to tackle this problem in the presence of large scale data input. Our approach utilises a novel superpixel method, specifically designed for hyperspectral data, to define meaningful local regions in an image, which with high probability share the same classification label. We then extract spectral and spatial features from these regions and use these to produce a contracted weighted graph-representation, where each node represents a region rather than a pixel. Our graph is then fed into a graph-based semi-supervised classifier which gives the final classification. We show that using superpixels in a graph representation is an effective tool for speeding up graphical classifiers applied to hyperspectral images. We demonstrate through exhaustive quantitative and qualitative results that our proposed method produces accurate classifications when an incredibly small amount of labelled data is used. We show that our approach mitigates the major drawbacks of existing approaches, resulting in our approach outperforming several comparative state-of-the-art techniques.

Figures

Tables

Table 1. TABLE I: The parameter values used for all experiments in this paper. Note that { x , y } 𝑥 𝑦 \{x,y\} signifies a random uniform distribution between x 𝑥 x and y 𝑦 y .

Fixed Parameters
Parameter	Description		Value
$m$	Controls the compactness of superpixels		10.0
$h$	Weighted filtering kernel		15.0
${\sigma }_s$	Kernel parameter for constructing $s_{ij}$		0.20
$k$	$k$-NN construction		8
$\mu$	Weighting in the LGC classifier		{0.1,0.15}
Data-based parameters
Parameter	Indian Pines	Salinas	Pavia Univerisity
$\beta$	0.9	0.9	0.1
${\sigma }_l$	{0.4,0.5}	{3.2.4.0}	{17,20}
$K$	$1200$	$1400$	$2400$

Table 2. TABLE II: OA (%) of Ten Repeated Experiments with Differing Numbers of training samples per class

Salinas
Samples per Class	OURS	LCMR [16]	SC-MK [13]	EPF [45]	LBP [15]	IFRF [14]	SVM [1]
3	98.0 $\pm$ 0.8%	83.7 $\pm$ 4.0%	82.8 $\pm$ 2.9%	78.2 $\pm$ 4.8%	78.5 $\pm$ 3.4%	87.7 $\pm$ 5.3%	76.6 $\pm$ 2.3%
5	99.1 $\pm$ 0.5%	89.9 $\pm$ 2.1%	85.0 $\pm$ 3.0%	80.9 $\pm$ 4.4%	84.8 $\pm$ 2.8%	93.1 $\pm$ 1.8%	79.6 $\pm$ 1.9%
7	99.1 $\pm$ 0.3%	92.3 $\pm$ 1.6%	88.2 $\pm$ 2.2%	84.9 $\pm$ 3.2%	88.4 $\pm$ 2.0%	93.8 $\pm$ 1.5%	81.3 $\pm$ 1.9%
10	99.0 $\pm$ 0.4%	93.8 $\pm$ 1.1%	90.1 $\pm$ 2.2%	86.4 $\pm$ 4.3%	91.4 $\pm$ 1.2%	95.4 $\pm$ 1.3%	82.4 $\pm$ 1.2%
15	99.1 $\pm$ 0.3%	94.7 $\pm$ 1.0%	93.1 $\pm$ 1.1%	89.2 $\pm$ 2.4%	93.0 $\pm$ 1.2%	97.1 $\pm$ 1.2%	84.3 $\pm$ 1.4%
20	99.3 $\pm$ 0.2%	96.1 $\pm$ 0.8%	93.3 $\pm$ 1.1%	89.8 $\pm$ 3.3%	95.1 $\pm$ 1.1%	97.3 $\pm$ 1.0%	84.5 $\pm$ 1.5%

Table 3. TABLE III: OA (%) AA (%) and Kappa (%) of ten consecutive experiments with ten training samples per class

Salinas
Technique	OURS	LCMR [16]	SC-MK [13]	EPF [45]	LBP [15]	IFRF [14]	SVM [1]
OA	99.24 $\pm$ 0.16%	93.90 $\pm$ 1.29%	90.38 $\pm$ 2.42%	86.53 $\pm$ 1.99%	90.68 $\pm$ 1.35%	95.87 $\pm$ 1.62%	82.42 $\pm$ 1.15%
AA	98.90 $\pm$ 1.51%	96.26 $\pm$ 1.02%	94.16 $\pm$ 1.11%	93.51 $\pm$ 0.91%	93.00 $\pm$ 1.03%	96.24 $\pm$ 1.43%	88.55 $\pm$ 0.99%
Kappa	99.15 $\pm$ 0.17%	93.22 $\pm$ 1.44%	89.33 $\pm$ 2.67%	85.10 $\pm$ 2.15%	90.68 $\pm$ 1.36%	95.41 $\pm$ 1.80%	80.53 $\pm$ 1.25%

Table 4. TABLE IV: Class breakdown of the three test data sets

Indian Pines				University of Pavia				Salinas
Class	Colour	Name	Number	Class	Colour	Name	Number	Class	Colour	Name	Number
1		Alfalfa	41	1		Asphalt	6621	1		Broccoli green weeds 1	2004
2		Corn-notill	1423	2		Meadows	18639	2		Broccoli green weeds 1	3721
3		Corn-mintill	825	3		Gravel	2089	3		Fallow	1971
4		Corn	232	4		Trees	3054	4		Fallow rough plow	1386
5		Grass-pasture	478	5		Metal sheet	1335	5		Fallow smooth	2673
6		Grass-trees	725	6		Bare soil	5019	6		Stubble	3954
7		Grass-pasture-mowed	24	7		Bitumen	1320	7		Celery	3574
8		Hay-windrowed	473	8		Bricks	3672	8		Grapes untrained	11266
9		Oats	15	9		Shadows	937	9		Soil vinyard develop	6198
10		Soybean-notill	967					10		Corn senesced green weeds	3273
11		Soybean-mintill	2450					11		Lettuce romaine 4wk	1063
12		Soybean-clean	588					12		Lettuce romaine 5wk	1922
13		Wheat	200					13		Lettuce romaine 6wk	911
14		Woods	1260					14		Lettuce romaine 7wk	1065
15		Buildings-Grass-Trees-Drives	381					15		Vinyard untrained	7263
16		Stone-Steel-Towers	88					16		Vinyard vertical trellis	1802

Table 5. TABLE V: OA(%) AA(%), Kappa and a class by class breakdown obtained by different classifiers with ten training samples per class. The best results are highlighted in green.

Indiana Pines
Class	SGL	LCMR [16]	SC-MK [13]	EPF [45]	LBP [15]	IFRF [14]	SVM [1]
1	98.04 $\pm$ 0.65%	99.44 $\pm$ 1.17%	98.33 $\pm$ 1.43%	53.78 $\pm$ 28.30%	100.0 $\pm$ 0%	53.51 $\pm$ 25.90%	20.53 $\pm$ 5.10%
2	76.06 $\pm$ 9.19%	75.99 $\pm$ 10.18%	78.82 $\pm$ 6.80%	55.88 $\pm$ 11.32%	70.94 $\pm$ 7.05%	70.35 $\pm$ 10.65%	43.14 $\pm$ 8.51%
3	84.27 $\pm$ 6.04%	70.78 $\pm$ 8.08%	77.65 $\pm$ 9.18%	60.19 $\pm$ 17.99%	70.28 $\pm$ 12.13%	67.99 $\pm$ 7.80%	39.15 $\pm$ 8.39%
4	96.67 $\pm$ 2.73%	93.88 $\pm$ 10,78%	86.39 $\pm$ 12.62%	34.57 $\pm$ 12.83%	97.93 $\pm$ 3.23%	79.37 $\pm$ 12.24%	21.25 $\pm$ 3.97%
5	92.30 $\pm$ 7.48%	90.32 $\pm$ 10.14%	82.33 $\pm$ 10.56%	93.39 $\pm$ 5.30%	82.92 $\pm$ 7.94%	79.40 $\pm$ 13.64%	59.34 $\pm$ 11.59%
6	98.71 $\pm$ 0.60%	91.40 $\pm$ 4.18%	89.54 $\pm$ 7.43%	86.32 $\pm$ 10.34%	90.36 $\pm$ 5.49%	93.63 $\pm$ 4.96%	83.29 $\pm$ 3.83%
7	100.0 $\pm$ 0.00%	100.0 $\pm$ 0.00%	100.0 $\pm$ 0.00%	70.92 $\pm$ 39.13%	100.0 $\pm$ 0.00%	39.65 $\pm$ 23.77%	24.85 $\pm$ 11.06%
8	100.0 $\pm$ 0.00%	99.68 $\pm$ 0.23%	97.09 $\pm$ 9.19%	98.41 $\pm$ 3.83%	100.0 $\pm$ 0.00%	99.97 $\pm$ 0.07%	93.12 $\pm$ 4.02%
9	100.0 $\pm$ 0.00%	100.0 $\pm$ 0.00%	100.0 $\pm$ 0.00%	59.44 $\pm$ 26.70%	100.0 $\pm$ 0.00%	28.81 $\pm$ 21.34%	12.59 $\pm$ 8.36%
10	88.94 $\pm$ 6.52%	76.46 $\pm$ 7.31%	71.32 $\pm$ 10.49%	61.19 $\pm$ 11.43%	79.90 $\pm$ 5.05%	75.95 $\pm$ 9.75%	36.93 $\pm$ 10.25%
11	91.04 $\pm$ 7.49%	71.40 $\pm$ 6.33%	69.22 $\pm$ 12.69%	81.05 $\pm$ 8.72%	73.78 $\pm$ 6.78%	93.81 $\pm$ 4.23%	61.50 $\pm$ 3.96%
12	90.05 $\pm$ 4.14%	90.50 $\pm$ 3.66%	78.47 $\pm$ 17.31%	44.31 $\pm$ 14.00%	70.58 $\pm$ 6.99%	74.08 $\pm$ 10.25%	28.20 $\pm$ 5.78%
13	99.56 $\pm$ 0.15%	99.33 $\pm$ 0.25%	99.90 $\pm$ 0.22%	98.34 $\pm$ 3.38%	98.31 $\pm$ 2.97%	75.32 $\pm$ 15.02%	80.12 $\pm$ 6.41%
14	100.0 $\pm$ 0.00%	98.18 $\pm$ 3.54%	88.14 $\pm$ 2.69%	95.15 $\pm$ 4.04%	91.32 $\pm$ 5.03%	98.31 $\pm$ 1.34%	88.22 $\pm$ 4.03%
15	97.69 $\pm$ 6.92%	91.62 $\pm$ 10.39%	90.96 $\pm$ 11.42%	64.91 $\pm$ 23.49%	90.59 $\pm$ 10.07%	77.14 $\pm$ 11.42%	39.10 $\pm$ 8.79%
16	100.0 $\pm$ 0.00%	98.67 $\pm$ 3.79%	97.59 $\pm$ 1.50%	84.46 $\pm$ 7.54%	98.43 $\pm$ 1.14%	92.62 $\pm$ 14.18%	87.71 $\pm$ 20.71%
OA	90.89 $\pm$ 2.98%	82.74 $\pm$ 2.32%	79.91 $\pm$ 2.60%	68.95 $\pm$ 2.01%	80.52 $\pm$ 2.03%	80.86 $\pm$ 3.76%	51.20 $\pm$ 3.92%
AA	92.16 $\pm$ 6.77%	90.48 $\pm$ 1.56%	87.86 $\pm$ 1.53%	71.39 $\pm$ 3.49%	88.46 $\pm$ 1.29%	74.99 $\pm$ 3.16%	51.19 $\pm$ 3.22%
Kappa	87.5 $\pm$ 3.33%	80.51 $\pm$ 2.59%	77.31 $\pm$ 2.93%	65.02 $\pm$ 2.25%	78.09 $\pm$ 2.23%	78.45 $\pm$ 4.16%	45.41 $\pm$ 4.11%

Equations

$$d(p_x,p_y)=||\text{logm}({\text{C}}_{p_x})-\text{logm}({\text{C}}_{p_y})|{|}_F$$

$$+||\hat{\mathbf{I}}(p_x)-\hat{\mathbf{I}}(p_y)||+\frac{m}{S}||p_x-p_y||.$$

$$j=\underset{s_j\in \mathcal{N}}{\mathrm{argmin}}||{\mathcal{P}}_i^m-{\mathcal{P}}_j^m||.$$

$${\vec{\mathcal{S}}}_i^m=\frac{{\sum }_{j=1}^{n_i}\hat{\mathbf{I}}(p_{i,j})}{n_i}.$$

$${\vec{\mathcal{S}}}_i^w=\sum _{j=1}^J{w}_{i,z_j}{\vec{\mathcal{S}}}_{z_j}^m,$$

$$w_{i,z_j}=\frac{\exp \left(-||{\vec{\mathcal{S}}}_{z_j}^m-{\vec{\mathcal{S}}}_i^m|{|}_2^2/h\right)}{{\sum }_{j=1}^J\exp \left(-||{\vec{\mathcal{S}}}_{z_j}^m-{\vec{\mathcal{S}}}_i^m|{|}_2^2/h\right)},$$

$${\vec{\mathcal{S}}}_i^p=\frac{{\sum }_{j=1}^{n_i}{p}_{i,j}}{n_i}.$$

$$w_{ij}=s_{ij}{l}_{ij},$$

$$s_{ij}=\exp \left(\frac{\left(\beta -1\right)||{\vec{\mathcal{S}}}_i^w-{\vec{\mathcal{S}}}_j^w|{|}_2^2-\beta ||{\vec{\mathcal{S}}}_i^m-{\vec{\mathcal{S}}}_j^m|{|}_2^2}{{\sigma }_s^2}\right),$$

$$l_{ij}=\exp \left(\frac{-||{\vec{\mathcal{S}}}_i^p-{\vec{\mathcal{S}}}_j^p|{|}_2^2}{{\sigma }_l^2}\right).$$

$$W_{ij}=\{\begin{array}{ll}{w}_{ij},& \text{if i is one of the k nearest neighbor of j,}\\ & \text{or vice versa.}\\ 0& \text{otherwise}.\end{array}$$

$$\mathcal{Q}(F)=\frac{1}{2}\sum _{i,j=1}^n{W}_{ij}{\left|\left|\frac{F_i}{\sqrt{D_{ii}}}-\frac{F_j}{\sqrt{D_{jj}}}\right|\right|}^2+\frac{\mu }{2}\sum _{i=1}^n||F_i-Y_i|{|}^2,$$

Full text

I Introduction

IN modern applications, hyperspectral images (HSI) capture a detailed light distribution, over several hundreds of spectral bands. This detailed spectral and spatial information increases the discriminative ability of HSIs compared to conventional colour images or multi-spectral images. As a result, hyperspectral imaging has been used in a wide range of applications including classification [1, 2, 3], object tracking [4, 5, 6], environmental monitoring [7, 8] and object detection [9, 10, 11].

In recent years, the classification of hyperspectral data has been an active topic of research. Classifying HSIs requires assigning a class label to each pixel within the image. There are several large hurdles to overcome during the classification process: the high dimensionality of the spectral information, the large spatial variability of the data and the limited number of training samples available due to the cost of data labelling. There have been numerous different attempts to deal with these problems when classifying HSIs, in which the majority of solutions rely on supervised learning (SL).

Kernel based classifiers such as support vector machines (SVM) [1, 12] are commonly used in the field. Whilst initial kernel methods only used spectral features, many later kernel methods included spatial features. An example being the multiple kernel learning (MKL) aproach of Fang et al [13] which used MKL to combine spatial based feature vectors alongside spectral features.

To deal with the high dimensionality of the data, many different feature extraction (FE) methods have been investigated. These methods aim at finding a lower dimensional subspace in which the separability among samples is maximised. Kang et al used image fusion and recursive filtering to extract meaningful features [14], Li at al [15] exploited local binary patterns to extract local features and textural information and Fang et al [16] used local co-variance matrix representation to characterize the correlation between the spectral and spatial information in HSI data.

Motivated by the remarkable success of Deep Learning (DL), different works have used DL for HSI classification. Convolutions neural networks (CNN) are commonly used to extract high level spectral and spatial features [17] [18]. Makantasis et al [17] used a CNN to extract spatial and spectral features and passed these into a multi-layer perceptron. In recent work, generative adversarial networks (GAN), which simultaneously train a generator and discriminator, have also been explored for HSI classification [19].

Although SL based classifiers have shown good results on HSI data, their performance is heavily reliant on having a large quality training set which is a costly investment. As an alternative to SL, we could use unsupervised learning (UL), in which the key idea is to rely on learning a set of classes from data that has not been labelled[20]. Although works such as [21] reported promising results on using UL for HSI classification, the major problem with UL is that the classification task becomes a massively ill-posed problem that needs specific assumptions to mitigate the lack of correspondence between the produced clusters and the known classes.

The aforementioned constraints associated with SL and UL make semi-supervised learning (SSL)[22] a clear alternative for obtaining an improved classification performance. The idea of SSL is to exploit both labelled and unlabelled data in the training process to produce a higher classification accuracy than solely using the labelled data. The advantages of SSL when using HSI data are two-fold: we decrease the need for large amounts of labelled data and we gain further understanding of the relationships present in the data.

In this paper, we introduce for the first time a superpixel contracted graph-based learning framework for semi-supervised HSI classification, that we named Superpixel Graph Learning (SGL). It produces state-of-the-art results, especially when the amount of labelled data is extremely small. Our framework is composed of three main parts. Firstly, we use a novel superpixel algorithm, specially designed for HSIs, to accurately partition our images into adaptive regions termed superpixels. Secondly, we perform feature extraction on each superpixel to extract discriminative features. Finally, we use the superpixels and features to produce a weighted graphical representation of our image which is then classified using a graphical-learning method (LGC [23]). Our main contributions are:

• •

We propose a novel computationally tractable framework for HSI classification, in which our novelty largely relies on:

• –

A hyperspectral superpixel approach. To the best of our knowledge, this is the first time that a superpixel approach has been designed specifically for HSI data, that is an approach which considers both spatial and spectral information. We proposed a new novel clustering distance, which combines a Euclidean spectral distance with the Log-Euclidean distance of a covariance matrix representation. This allows us to define meaningful local regions to boost the overall classification performance.

• –

Superpixel graph classification. We show that combining superpixels with a graphical representation and a purely graphical classifier brings two major advantages: firstly, it vastly decreases the size of the node set which allows for classification in computationally feasible times without the need for matrix approximation methods. Secondly, it allows for the intelligent regularisation of the final classification map by using superpixels as adaptive local regions.

• •

We extensively validate our proposed approach by using three benchmarking datasets and provide a range of experimental results.

• •

We demonstrate that the combination of our novel hyperspectral superpixel approach embedded in a graphical setting leads to state of the art results for HSI classification.

The remainder of this paper is organised as follows. Section II explores the related work on semi-supervised learning in the context of HSI classification. Section III is devoted to describing the proposed SGL method including superpixel generation, feature extraction and graph-based semi supervised classification. Section IV contains the experimental results for testing upon three real HSIs and a comparison to other state-of-the-art classification methods. Finally, Section V presents the conclusions as well as discussion of further work.

II Related Work

The problem of semi-supervised classification of HSIs has been previously investigated by the remote sensing community. In this section, we review the existing techniques in turn. The literature regarding Semi-Supervised Learning (SSL) algorithms can be roughly categorised into three different categories. These being generative models, low-density separation and graph-based methods.

Several previous methods have utilised graph-based learning, and our paper is closely related to these. Graph-based methods rely upon constructing a graph representation, where the data points are represented by nodes and the similarity between these data points shown by edges and weights (see Fig 1). The first graph-based learning method was proposed by Camps-Valls in [24]. This paper used different spectral and spatial kernels alongside the Nyström extension, as a matrix approximation tool, to classify HSIs in computationally reasonable times. However, the produced accuracy was poor compared to other methods at the time. Gao et al [25] used a bilayer graph-based learning algorithm to improve classification performance. The two layers were composed of a pixel-based graph, similar to [24], and a hypergraph built from grouping relations estimated using unsupervised learning. Cui et al [26] used an extended random walker (ERW) on a superpixel-based graph to optimise a classification map produced from an SVM. Showing that the accuracy of the SVM could be greatly improved by using the information present in the graph.

Another group of semi-supervised methods seek to directly implement the low density separation assumption [22] by moving the decision boundary away from unlabeled points. The first paper published in this area was by Bruzzone et al [27] which used a novel transductive SVM (TSVM) for HSI classification. A TSVM differs from the standard SVM as it seeks to maximise the margin on a combination of labeled and unlabeled data. Building upon these ideas came semi-supervised self-learning algorithms such as the work by Dópido et al [28], in which they sought to adapt active learning, in a which a user actively selects unlabeled samples, to a self learning framework in which the computer automatically selects the most informative unlabeled samples for classification purposes. Ratle et al [29] took a different path and tackled low density separation using a semi-supervised neural network architecture. An embedding regularizer was added to the loss function to inject the unlabeled information and this approach produced higher classification accuracy than TSVMs.

The rise of deep learning methods, has led to an increase in popularity of generative methods for semi-supervised learning. However, these methods are in still in their infancy. One of the most popular approaches by Zhan et al [30] uses a generative adversarial network (GAN) to simultaneous train a discriminator and generator. However, this paper uses a 1D-GAN and can only exploit spectral feature and the produced accuracy suffers as a result. Zhu et al [19] developed a 3D-GAN which used convolutional neural networks for the discriminator and generator. This architecture allows the approach to exploit the spectral-spatial information present in the HSI. Therefore, the produced accuracy was much higher than [30].

Although works based on generative models and low-density separation have shown encouraging results, in this work, we concentrate on producing a graph-based method, the motivation for which is three-fold. Firstly, data can be naturally represented on graphs. Secondly, a graph representation is motivated by its mathematical background and properties including spareness. Thirdly, data can be represented in an uniform space even if the data is highly heterogeneous. We seek to produce a graph-based method that is based on superpixel representations similar to that of [26]. However, unlike [26] we seek to produce a fully graph-based learning method rather than a graph-based optimisation of a non-graph based method.

III Proposed Method

This section is devoted to explaining our proposed framework, which we call SGL. It contains three main parts which are shown in Fig 2. Firstly we describe our hyperspectral superpixel algorithm, subsequently we give a description of the feature extraction process and finally we describe the construction and classification of our graph representation.

Problem Statement. In this work, we seek to find an accurate classification prediction for a large amount of unlabelled data given an extremely small amount of labelled data. We consider the following problem definition for the classification task under the SSL paradigm.

Definition 1

Semi-supervised Classification Task. Given a set of points $\{(x_{i},y_{i})\}_{i=1}^{l}$, $\{x_{k}\}_{k=l+1}^{l+u}$, and a label set $\mathcal{L}=\{1,..,c\}$ where $\{y_{i}\}_{i=1}^{l}\in\mathcal{L}$, then, we seek to find a function $f:\mathcal{X}^{l+u}\mapsto\mathcal{Y}^{l+u}$, which utilises the unlabelled data $\{x_{k}\}_{k=l+1}^{l+u}$, such that $f$ allows for a good prediction for $\{x_{k}\}_{k=l+1}^{l+u}$.

III-A Superpixel Segmentation

Superpixels are perceptually meaningful connected regions which group pixels similar in colour or other features and were initially introduced by Ren and Malik [31]. In subsequent years, many different algorithmic approaches have been proposed (e.g.[32, 33, 34]). For a detailed survey on superpixel algorithms see [35]. Fig 3 shows the application of a superpixel algorithm to a HSI. Superpixel maps such as the one shown in Fig 3 have many desirable properties: they are computationally and representationally efficient, the individual superpixels are perceptually meaningful and as superpixels are the result of an over-segmentation they are very good at conserving image structures.

Why use superpixels as a tool for HSIs? In order to extract spatial features for use in spectral-spatial models, it is important to be able to define good local regions. Whilst setting a fixed size window (e.g. [36]) has shown good results, a fixed size does not allow for the full exploitation of spatial context. Using superpixels as adaptive regions [13] has been shown to produce discriminative information. Cui et al. [26] demonstrated this by using a superpixel based random walker to optimise an SVM probability map to great effect. Furthermore, Cui et al. additionally demonstrated that a superpixel spectrum is more stable and less affected by noise that an individual pixel spectrum. Therefore by using superpixels we become more resistant to noise present in the data.

The most common algorithm used in clustering based superpixel methods is Lloyd’s algorithm [37], a modified version of the popular k-means clustering algorithm. In the context of Lloyd’s algorithm, let us first formalise the definition of a superpixel segmentation.

Definition 2

Superpixel Segmentation. Given an image $I:\mathcal{X}\rightarrow\Omega$, where $\mathcal{X}\subset\mathbb{Z}^{2}$, a superpixel over-segmentation is a partition $\{\mathcal{S}_{i}\}_{i=1}^{n}$ of $\mathcal{X}$ such that for each $1\leq i\leq n$ we have $S_{i}=\{x:d((x,I(x)),F(S_{i}))$, where $d$ is a metric, $F$ is a feature function and $\mathcal{S}_{i}$ is an individual superpixel.

In this work, we build on this definition to propose our algorithmic approach. Denote a HSI as $\textbf{I}=\{I_{b}\},b=1,..,\mathcal{B}$ with dimensions $\mathcal{W}\times\mathcal{H}\times\mathcal{B}$ representing the width, height and number of bands respectively. Firstly, for computational efficiency we use PCA [38] to reduce the dimensionality and produce a reduced image $\mathbf{\hat{I}}=\{\hat{I}_{a}\},a=1,..,\mathcal{A}$ where $\mathcal{A}\ll\mathcal{B}$. Denoting an individual pixel as $p\in\mathbf{\hat{I}}$, we then seek to partition our reduced HSI $\mathbf{\hat{I}}$ into superpixels. This corresponds to splitting $\mathbf{\hat{I}}$ into a family of disjoint sets, $\mathbf{\hat{I}}=\cup_{i=1}^{K}\mathcal{S}_{i}$ , $\mathcal{S}_{i}\cap\mathcal{S}_{j}=\emptyset$, where $\mathcal{S}_{i}$ corresponds to an individual superpixel and $K$ is the number of superpixels. Each superpixel $\mathcal{S}_{i}$ is made up of a set of $n_{i}$ connected pixels, $\mathcal{S}_{i}=\{p_{i,1},...,p_{i,n_{i}}\}$.

Hyperspectral superpixel construction. When constructing our hyperspectral superpixels, we need to ensure that our algorithm extracts effective information from hyperspectral data. Whilst other works, including [13], feed the first three principal components of HSIs into RGB based superpixel algorithms, we seek to design an algorithm specifically built for hyperspectral data to ensure good performance.

As the base for our algorithm, we began with Manifold SLIC (MSLIC) [33]. MSLIC has two features that make it highly useful for our purpose. Firstly it produces content sensitive superpixels by mapping the image $I$ to a two dimensional manifold $\mathcal{M}$ and measuring the area of Voronoi cells on $\mathcal{M}$. Secondly, the number of superpixels will change from the initial selection to fit the content structure in the image, thereby lowering the chances of a poor initial choice of $K$ greatly reducing the final accuracy.

Our proposed method is an novel extension of MSLIC into hyperspectral data. We name this extension Hyper Manifold SLIC (HMS). HMS involves three major changes over MSLIC.

III-A1 High dimensional adaption

We alter the MSLIC algorithm to take image data with any number of bands $\mathcal{B}$. This involves changing several steps such as mapping the image $\mathbf{{I}}$ to a 2-dimensional manifold $\mathcal{M}\in\mathbb{R}^{\mathcal{B}+2}$ rather than the standard $\mathcal{M}\in\mathbb{R}^{5}$.

III-A2 Hyperspectral clustering distance

Based on our previous work [39], we design a more effective clustering distance as a combination of the Euclidean spectral distance [34] and Log-Euclidean (LED) distance [40] of a covariance matrix representation [41]. This combination effectively combines the spatial and spectral data present in the image. For each pixel ${p}\in\mathbf{\hat{I}}$ we construct a covariance matrix $\textbf{C}_{{p}}$ using the same methodology as Fang et al [16] and use the LED metric to calculate the distances between these matrices. The distance between two pixels $p_{x}$, $p_{y}$ is given by:

[TABLE]

From (1), the parameter $m$ controls the compactness of superpixels whilst $S$ scales the spatial distance and, for a image with $N$ pixels, $S=\sqrt{N/K}$ as in the MSLIC algorithm.

III-A3 Spectral Merging

In the original MSLIC algorithm when the area of a seed $s_{i}$ is below a threshold it is randomly merged with a neighbouring seed $s_{j}$. However, in our implementation we instead choose the neighbouring seed which satisfies:

[TABLE]

where $\mathcal{P}^{m}_{i}$ is the average spectral information of the seed $s_{i}$ and $\mathcal{N}$ is the set of neighbouring seeds. We choose to merge superpixels, which are most similar in their spectral properties, as this yields a better form of adaptation to the hyperspectral data.

These proposed changes produce accurate superpixels for HSIs. For further details on our approach, refer to Section II of the supplementary material.

III-B Feature Extraction

Now we seek to extract meaningful features from the extracted superpixels ready for graph construction. In this paper, we use the same features as we did in our previous work on superpixels [39]. From each superpixel $\mathcal{S}_{i}$ we extract three different features. To extract localised spatial information we apply a mean filter to each superpixel to produce a mean feature vector $\vec{\mathcal{S}}^{m}_{i}$ which is defined as .

[TABLE]

Using a weighted combination of the mean feature vectors of a superpixel’s adjacent neighbours, we can obtain a measure of the spatial information between superpixels. Note that adjacency is defined based on 4-connectivity on the image grid. For each superpixel $\mathcal{S}_{i}$, we define the set $\mathcal{Z}_{i}=\{z_{1},z_{2}..,z_{J}\}$ which contains the $J$ indexes of its adjacent superpixels. From this, we construct the weighted feature vector $\vec{\mathcal{S}}_{i}^{w}$ which reads:

[TABLE]

where the weight between adjacent superpixels $w_{i,z_{j}}$ is defined as:

[TABLE]

with $h$ as a predefined scalar parameter. Finally, we propose to extract the centroidal location of each superpixel $\vec{\mathcal{S}}^{p}_{i}$ which we calculate as:

[TABLE]

III-C Graph based Classification

After defining how to get our superpixel set and extracted features, we now turn to explain how we create our weighted graph-representation. However, we first give some background into challenges associated with the computational implementation of graph-based methods and how superpixels can be used to overcome some of these.

As noted by Camps-Valls et al [42], many graphical algorithms rely on calculating and manipulating large kernel matrices formed by the labelled and unlabelled data. As an example, for an image with $n$ pixels the associated graph Laplacian is a matrix of size $n\times n$. If we seek to inverse the graph Laplacian via singular value decomposition then the computational complexity would be $O(n^{3})$, ruining the scaling that we seek.

Approximation methods do exist to speed up such matrix inversions. One commonly used technique is the Nyström extension [43] and it is regularly used to speed up matrix calculations [42] [44]. However, the Nyström extension has several drawbacks. It is unsuitable for sparse applications as the Nyström extension acts as an approximation for complete matrices.

In this paper, we implement a novel solution to increase the speed and reduce the complexity of graphical classifiers applied to HSIs. Instead of having a graphical representation where each node represents a pixel, we instead use our segmented superpixels as the node set. This greatly reduces the size of our node set as $K\ll n$ and allows us to perform matrix inversion and other calculations without approximations such as the Nyström extension. Furthermore, a superpixel representation should help to boost the classification accuracy as we are defining strong local regions in our data. Therefore, from these previously discussed features and our superpixel node set, a weighted, undirected graph $G=(V,E,W)$ can be created. The weight between two connected superpixels $\mathcal{S}_{i}$ and $\mathcal{S}_{j}$ is constructed based on two Gaussian kernels and is given as

[TABLE]

where

[TABLE]

[TABLE]

where $\beta$ balances the influence between the mean and weighted features and $\sigma_{s},\sigma_{l}$ determine the width of the Gaussian kernels. Note that weights are limited in value between $[0,1]$ with $1$ implying most similar. The edge set is constructed using $k$-nearest neighbours. Therefore, the edge weights are defined as:

[TABLE]

In the training stage of the algorithm, a set of labelled spectral pixels are randomly selected from the original HSI. The initial label of each superpixel is taken as the average initial label of its corresponding set of pixels. If no pixel within a superpixel is initially labelled then the superpixel is initially unlabelled. The labelling information for the superpixels are specified using a matrix $Y\in\mathbb{R}^{K\times c}$, where $c$ is the number of classes present and $K$ is the number of superpixels. $Y_{vl}$ specifies the value of the seed label $l$ for node $v$. The weight matrix and the initial labelling are then passed into Local and Global Consistency (LCG) algorithm [23].

LCG is a graph based SSL approach that formalises the smoothness and clustering assumptions of semi-supervised learning by designing a classification function which is smooth upon the graphical structure generated by all the data. The final labelling is specified using a matrix $F\in\mathbb{R}^{K\times c}$. The cost function associated with the matrix $F$ is given by

[TABLE]

where $\mu>0$ is a regularisation parameter. $\mathcal{F}$ denotes the set of $n\times c$ matrices with non-negative entries. The labelling matrix is given by $F^{*}=\operatorname*{argmin}_{F\in\mathcal{F}}\mathcal{Q}(F)$.

The first term in the cost function is the smoothess constraint, which encourages connected nodes to have similar labelling, whilst the second term fits the finally labelling to the initially labelled data. Balance between these constraints is set by the parameter $\mu$. The above cost function has a closed form solution which reads: $F^{*}=\beta\left(I-\alpha D^{-\frac{1}{2}}WD^{-\frac{1}{2}}\right)Y$, where $\beta=\frac{\mu}{1+\mu}$ and $\alpha=1-\beta$. The final labelling of the nodes is then computed as: $y_{i}=\operatorname*{argmax}_{j\leq c}F_{ij}$. The superpixel labels and the superpixel segmentation are used to construct the final pixel classification map.

IV Experimental Results

In this section, we detail the experiments conducted to validate the proposed approach.

IV-A Data Description

We use three benchmark HSI datasets to evaluate our approach, which have the following characteristics.

• •

Indian Pines Dataset. The dataset was collected by an airborne visible/infrared imaging spectrometer (AVIRIS) sensor over an agricultural site in Indiana and has 16 classes. The data set consists of $145\times 145$ pixels, $200$ spectral channels, a spectral range of $0.4$ to $2.5\mu$m and a spatial resolution of $20$m.

• •

Salinas. This image was also collected by the AVIRIS sensor over Salinas Valley, California, and contains 16 classes. The data set size is $512\times 217$ pixels and identical to Indian Pines has $200$ spectral channels over $0.4$ to $2.5\mu$m. The data set is characterised by a high spatial resolution pf $3.7$m per pixel.

• •

University of Pavia. This dataset was acquired by the reflective optics system imaging spectrometer (ROSIS). The image ($610\times 340$ pixels) covers the Engineering School at the University of Pavia and has 9 classes. The image contains 115 spectral channels from $0.43$ to $0.86\mu$m and has a has a spatial resolution of $1.3$m.

In section III of the supplementary material, we further describe the datasets used and the mathematical background of the evaluation criteria.

IV-B Evaluation Protocol

For all experiments carried out in this paper, each one is repeated $10$ times and the average and standard deviation are provided for each measurement. The number of principal components used were set by demanding that the total explained variance ratio was $\geq 0.999$. To evaluate the performance of each HSI classifier, we use three commonly implemented evaluation criteria Overall Accuracy (OA), Average Accuracy (AA) and the Kappa Coefficient.

To validate the performance of our proposed classification framework SGL, several state-of-the-art HSI classification methods have been implemented to act as comparisons. These are local co-variance matrix representation (LCMR) [16], superpixel-based classification via multiple kernels (SC-MK) [13], the edge preserving filter based method (EPF) [45], local binary patterns (LBP) [15], an SVM method [1] and image fusion and recursive filtering (IFRF) [14].

IV-C Parameter Selection

In our proposed framework, there are eight hyperparameters that come from the four tasks of our framework.

• Superpixel construction: $K$ and $m$.
• Feature Extraction: $h$.
• Graph construction: $\sigma_{s}$, $\sigma_{l}$, $k$ and $\beta$.
• LGC classification: $\mu$.

For the superpixels construction step, we set the ratio of the number of pixels to the number of superpixels $\frac{N}{K}$ must be at least $15$. The parameters $m$, $h$, $\sigma_{s}$, $k$ and $\mu$ have the same value for all datasets used. These values were found using empirical testing in a coarse to fine search method. The other three parameters, $\sigma_{l}$, $\beta$ and $K$, change value depending on the HSI used. The parameter values used in the experiments are given in Table I.

We leave a discussion of the parameters $\beta$ and $\sigma_{l}$ to section III of the supplementary material and focus here on the superpixel number $K$. Given that we are using a superpixel based classifier, it is critically important to understand how the superpixel number $K$ effects the accuracy. This is especially true when it is unclear what value of $K$ to pick for a given image. To investigate the effect of changing the parameter $K$, we classified all three HSIs using a varying number of superpixels and $7$ randomly selected samples from each class and plotted the classification accuracy against the superpixel number. The results for this analysis are given in Fig. 4. In general the classification accuracy increases with the number of superpixels, due to the underlying over-segmentation being more accurate. However, once the image is accurately over-segmented, there are diminishing returns for further increasing the superpixel number. Combined with the fact that increasing the number of superpixels increases the size of the graph and thus the running time, we used the smallest number superpixels that reliably gave a good classification accuracy for each HSI.

For the compared methods the parameters were set using the default values provided in the demo code or referenced in the papers themselves. The SVM method was implemented using the LIBSVM [46] library and uses a Gaussian kernel and five-fold cross validation.

IV-D Experimental Results

Our experiments are organised into two parts. Firstly, we compare the classification accuracy of our proposed framework with the comparison classifiers detailed above. Due to the semi-supervised nature of our method, we will be testing the classification performance using very limited amounts of training data. Secondly, we will seek to use visual classification maps to understand and explain the performance of our classifier to relation to the other classifiers.

(E1) In our first experiment, we evaluate the overall accuracy (OA) of our method against the state-of-the-art when using a reduced amount of labelled data for training ($\{3,5,7,10,15,20\}$ randomly selected samples per class). The accuracy of the different classifiers for the three benchmark datasets are given in Table II and the graphical representation of the results is shown in Fig. 5.

We see that the accuracy produced by the SGL framework is, by a significant margin, the best of any classifier considered in this paper. The SGL framework produces the best accuracy for all three benchmark images for each differing amount of labelled data. In particular, the average difference in OA between SGL and its nearest competitor LCMR [16], across the three datasets, was $9\%$ when using $5$ samples per class and was $13.7\%$ when using $3$ samples per class. Highlighting the semi-supervised nature of the SGL framework that allows it to exploit information present in the unlabelled data to overcome the limited amount of labelled samples.

(E2) To gain an understanding about how each classifier was performing and the explanation for the large increase in classification accuracy obtained by SGL, we produce visual classification maps. For each HSI we use ten labelled samples per class and calculated the overall accuracy (OA), average accuracy (AA), the Kappa coefficient and the full classification map. The results for this experiment are reported in Table III. Furthermore, Fig. 6-8 give a colour composite image, ground truth image and the final classification maps for the seven considered methods. In section III of the supplementary material we provide a class by class accuracy breakdown.

Examining the OA, AA and Kappa coefficient of the differing methods, we observe that SGL is again the best performing method with an average improvement of OA $+12.3\%$, AA $+9.5\%$ and Kappa $+11.6\%$ in the Indian pines scene, OA $+15.4\%$, AA $+13.8\%$ and Kappa $+17.3\%$ in the Pavia University scene and OA $+7.8\%$, AA $+4.3\%$ and Kappa $+8.4\%$ in the Salinas scene compared to the other classifiers (excluding the SVM).

To provide an explanation for the fantastic performance of SGL compared to the other methods let us examine the classification maps. The poorest performing classifier was the SVM. The SVM method only uses spectral information and as a result produces very noisy classification maps. The EPF method seeks to optimise the SVM classification map with an edge preserving filter to smooth out some of this noise and from these results we can see it successfully does so. However, the poor performance of the underlying SVM classification prevents the EPF method from achieving good classification. The LBP and IFRF methods produce over-smooth classification results when only a limited amount of data is available. This causes poor performance in the more complicated Indian Pines and Pavia University images. The LCMR and SCMK methods are the closest competitors to the SGL method with LCMR slightly outperforming the SCMK method due to a slightly higher amount of smoothing. Both of these methods manage to preserve edges and boundaries whilst producing smooth classification maps. This is due to the inclusion of spatial information via local neighbouring pixel construction and superpixel based kernels respectively.

What sets SGL apart from the other methods considered is that the classification map has been intelligently smoothed with near complete preservation of edges and boundaries. Primarily, this is due to the use of superpixels as the node set in our graph. The superpixels produce by our novel superpixel algorithm have accurately preserved the edges and boundaries in the image. Therefore, when we assign labels to each superpixel, rather than each pixel, we smooth our classification map across the homogeneous superpixels whilst retaining boundaries.

V Conclusion

In this paper, we have developed a novel semi-semi-supervised graph-based approach, SGL, for the classification of hyperspectral images. The proposed method can be split into three main stages: over-segmentation of HSIs with a novel superpixel algorithm specially designed for dealing with hyperspectral data, feature extraction to extract discriminative features and graph construction and classification. Our experiments with real benchmark HSIs demonstrate that our proposed method greatly outperforms other state-of-the-art classifiers in terms of qualitative and quantitative results, especially when using an incredibly small amount of data.

The semi-supervised nature of our solution exploits data present in the unlabelled data and can overcome the issue of having a highly limited training set, a common problem in the field of remote sensing. Furthermore, for the first time we propose using superpixels as the nodes of a pure graphical classifier which has two large benefits. Firstly, the size of the superpixel graph is much smaller than a pixel based graph allowing for computational reasonable run times without the need for matrix approximations. Secondly, applying labels to superpixels intelligent smooths our classification maps with near perfect preservation of edges and boundaries.

In our future work , we intend on applying deep learning to automate the extraction of deep features. Furthermore, we seek to apply recent work on heterogeneous graphs to investigate a combined superpixel/pixel representation.

VI Acknowledgment

The authors would like to thank Prof. D. Landgrebe from Purdue University and the NASA Jet Propulsion Laboratory for providing the hyperspectral data sets. We would also like to thank Prof David Coomes from the Department of Plant Sciences, University of Cambridge, for his advice and support. This work was supported by the UK Engineering and Physical Sciences Research Council (EPSRC) and the National Physical Laboratory (NPL). Support from the Centre for Mathematical Imaging in Healthcare (CMIH) University of Cambridge and the Maths in Healthcare Centre is greatly acknowledged. CBS acknowledges support from the Leverhulme Trust project on Breaking the non-convexity barrier, the Philip Leverhulme Prize, the EPSRC grant Nr. EP/M00483X/1, the EPSRC Centre Nr. EP/N014588/1, the European Union Horizon 2020 research and innovation programmes under the Marie Skodowska-Curie grant agreement No 777826 NoMADS and No 691070 CHiPS, the Cantab Capital Institute for the Mathematics of Information and the Alan Turing Institute. We gratefully acknowledge the support of NVIDIA Corporation with the donation of a Quadro P6000 GPU used for this research.

I Outline

The purpose of this supplementary material is to provide further details of the methodology used in the main paper as well as provide additional experimental results to validate the performance of our proposed framework. The supplementary material is divided into two sections:

• •

Section II. In this section we give more detail into the hyperspectral extension of MSLIC [33], named Hyper-Manifold SLIC (HSM), which allows it to accurately over-segment hyperspectral images.

• •

Section III. In this section we describe the benchmarking datasets in additional detail, further expand the parameter analysis, and provide additional classification results of the SGL method.

II Superpixel Construction

As our starting point, we used the Manifold SLIC method [33] algorithm developed by Liu et al and we would refer readers to their paper for a detailed explanation of the manifold extension. In the main paper, we list the major changes we made to the MSLIC algorithm to produce our hyperspectral extension which we call Hyperspectral Manifold SLIC (HMS). In this section, we list some other additional changes that we made to MSLIC that were not stated in the main paper. We also include additional visual examples of the application of HMS to real HSIs.

II-A Parameter Changes

From the original paper we made a small number of parameter changes. In this section, we give the reasoning behind these changes.

Convergence Conditions. In our implementation we stop the iteration loop when either the residual energy increases or when the percentage decrease in the residual energy is less than $10\%$. This is to prevent the superpixel algorithm for running for extended periods of time.

Enforced Connectivity. The final superpixels generated by the algorithm must be 4-connected on the image grid. This is ensured by an enforced connectivity algorithm which has two thresholds for the minimum and maximum superpixel size. In our algorithm, we use a minimum superpixel size of 8 and a maximum superpixel size of $\frac{10n}{K}$, where $n$ is the number of pixels and $K$ is the initial number of superpixels. The reason for choosing such a small minimum cluster size was that certain classes in the Indian Pines dataset were tiny in size and we needed to be able to capture these small areas.

II-B Supplementary Visual Results for HSM

In Figs 9-11 we provide more examples of the application of HSM to the three different HSIs considered in the main paper. We provide over-segmentations with differing numbers of superpixels highlighting the content sensitivity of the algorithm. In particular, note that Indian Pines and Salinas are easily over-segmented using a small number of superpixels whilst the more complex structure of Pavia University requires more superpixels to achieve an accurate over-segmentation.

III Supplementary Results

In this section, we expand the details regarding the experimental methodology used. Additionally, we provide further experimental results that validate the performance of our proposed framework SGL.

III-A Further Description of the Data Sets

The three labelled datasets used in the main paper are ”AVIRIS Indian Pines”, ”AVIRIS Salinas” and ”Reflective Optics System Imaging Spectrometer (ROSIS-03) University of Pavia.” Whilst the main paper describes the format of the three datasets, In this section, we give further details on the data sets and prepossessing. A class by class breakdown of each data set listing the different classes and the number of samples is given in Table IV.

Indian Pines consists of mainly agricultural classes with a small amount of non-organic land cover. Due to presence The different classes vary greatly in size with the smallest classes in the tens of pixels whilst the largest classes have several thousand pixels. Bands [104-108], [150-163] and 220 were removed prior to classification due to water absorption effects.

Salinas is made up entirely of 16 different vegetation classes. The classes are larger with the smallest class comprising several hundred pixels. The scene has two large classes: ”grapes untrained” and ”vineyard untrained” which dominate a large area of land cover. We remove bands [108-112], [154-167] and 224 due to water absorption effects.

University of Pavia is different from the other benchmarks in that it is contains a significant amount of non-organic land cover such as asphalt and bricks. This scene contains a small number of classes and a more complex geometry which should make it harder for a superpixel based classifier to classify.

III-B Description of the performance metrics

In the main paper, we use three commonly used evaluation criteria to evaluate the performance of each classifier. In this section, we give the explicit description of each of these criteria.

Overall Accuracy (OA). This measure is the ratio of the number of correctly classified pixels divided by the total number of pixels.

Average Accuracy (AA). This measure gives the average classification accuracy of all classes in an image.

Kappa Coefficient. This metric gives the agreement between the final classification and the ground-truth. It gives the percentage agreement corrected by the chance that this agreement is due to chance alone and is thought to be more robust that simple percentage agreement.

III-C Parameter Analysis

In the main paper, we explained how the classification accuracy changed with the parameter $K$. In this section we explain how the parameters $\sigma_{l}$ and $\beta$ change value depending on the HSI used.

$\sigma_{l}$ is the deviation of the location based Gaussian kernel $l_{ij}$. Consider increasing the image size whilst keeping $\sigma_{l}$ constant. The width of the Gaussian distribution would become narrower and narrower with respect to the size of the image. This would reduce the weight of the edges connecting superpixels that are further apart compared to superpixel pairs which are close. Eventually the decreasing width of the kernel would lead to the removal of all non-local connections in the graph preventing information from properly propagating across the graph. Therefore, to balance the width of the spatial kernel $l_{ij}$ to the spectral kernel $s_{ij}$, which does not change with image size, the value of $\sigma_{l}$ should increase.

$\beta$ weights between the two different spatial-spectral features $\vec{\mathcal{S}}^{m}_{i}$ and $\vec{\mathcal{S}}^{w}_{i}$ , with a lower value favouring the mean filter whilst a higher value emphasises the weighted filter. It was found that mean filtering was more effective for classifying Pavia University whilst weighted filtering was more effective when classifying Indian Pines and Salinas. An initial explanation for this is that the more complex land cover structure of the Pavia University scene means that the spatial information between superpixels is less informative than the spatial information within a superpixel.

III-D Further Experimental Results

(E3) As an additional experiment, we classified each HSI using the same parameters as the main paper with $10$ labelled samples per class. From this, we produced a class by class accuracy breakdown. The results for this experiment are contained in Tables V. From these tables, we see that the SGL method produced the highest classification accuracy for the majority of individual classes in each HSI with particular dominance in the Salinas and Indian Pines images. For the Indian Pines scene SGL produced clear accuracy improvements for classes 3 (Corn-mintill) and 10 (Soybean-mintill) in particular. Similarly, in the Salinas scene, SGL produced large improvements in the classification of classes 8 (Grapes untrained) and 15 (Vinyard untrained) as could be seen from the visual classificaiton maps in the main paper.