A scalable saliency-based Feature selection method with instance level information

TL;DR

This paper introduces a scalable saliency-based feature selection method that leverages deep learning saliency techniques to identify relevant features at the instance level, improving interpretability for specific samples.

Contribution

The paper presents a novel feature selection approach, Saliency-based Feature Selection (SFS), applicable across neural network architectures and trained with gradient descent, focusing on instance-specific relevance.

Findings

Effective in neural networks and other gradient descent-trained models

Provides instance-level feature relevance information

Enhances interpretability of models for specific samples

Abstract

Classic feature selection techniques remove those features that are either irrelevant or redundant, achieving a subset of relevant features that help to provide a better knowledge extraction. This allows the creation of compact models that are easier to interpret. Most of these techniques work over the whole dataset, but they are unable to provide the user with successful information when only instance information is needed. In short, given any example, classic feature selection algorithms do not give any information about which the most relevant information is, regarding this sample. This work aims to overcome this handicap by developing a novel feature selection method, called Saliency-based Feature Selection (SFS), based in deep-learning saliency techniques. Our experimental results will prove that this algorithm can be successfully used not only in Neural Networks, but also under any given architecture trained by using Gradient Descent techniques.

Figures

Equations

$$\sum _{c=1}^C{y}_c^{(i)}=1,$$

$$y_c^{(i)}=\text{mbox}softmax({\theta }_c^T{\mathbf{X}}^{(i)}).$$

$$\ell (\mathbf{\Theta };f,\mathbf{X},\mathbf{Y})=-\frac{1}{N}\sum _{i=1}^N\sum _{c=1}^C{y}_c^{(i)}\log \left(f({\mathbf{x}}^{(i)};\mathbf{\Theta }{)}_c\right)$$

$$\sigma^{(i)}_{c}=\bigg{|}\dfrac{\partial y_{c}^{(i)}}{\partial\mathbf{X}^{(i)}}\bigg{|}.$$

$$\sigma ({\tilde{y}}^{(i)},y^{(i)})=\left|\frac{\partial \text{g}({\tilde{y}}^{(i)})}{\partial {\mathbf{x}}^{(i)}}\right|,$$

$${\ell }_{MSE}(\tilde{\mathbf{Y}},\mathbf{Y})=-\frac{1}{N}\sum _{i=1}^N{\left({\tilde{\mathbf{y}}}^{(i)}-{\mathbf{y}}^{(i)}\right)}^2,$$

$$g_{MSE}(\tilde{\mathbf{Y}},\mathbf{Y})=\frac{\alpha }{{\ell }_{MSE}(\tilde{\mathbf{Y}},\mathbf{Y}),+ϵ},$$

$$g_{CE}(\tilde{\mathbf{Y}},\mathbf{Y})=-\frac{\alpha }{N}\sum _{i=1}^N\sum _{c=1}^C{y}_c^{(i)}\log \left(1-{\hat{y}}_c^{(i)}\right),$$

$${\hat{y}}_c^{(i)}=\min \left\{1-ϵ,{\tilde{y}}_c^{(i)}\right\},$$

$${\ell }_H(\tilde{\mathbf{Y}},\mathbf{Y})=-\frac{1}{N}\sum _{i=1}^N\sum _{c=1}^C{y}_c^{(i)}\max (0,1-{\tilde{y}}_c^{(i)})+(1-y_c^{(i)})\max (0,1+{\tilde{y}}_c^{(i)}),$$

$$g_H(\tilde{\mathbf{Y}},\mathbf{Y})=-\frac{\alpha }{N}\sum _{i=1}^N\sum _{c=1}^C{y}_c^{(i)}\log \left(1-{\breve{y}}_c^{(i)}\right),$$

$${\breve{y}}_c^{(i)}=\min \left\{1-ϵ,\frac{\min (1,\max (-1,{\tilde{y}}_c^{(i)}))+1}{2}\right\}.$$

Full text

1 Introduction

With the rise of the so-called Big Data, there is an increasing need for the use of techniques that allow us to reduce the input space [1]. These techniques are often divided into two broad groups [2]: Feature Selection (FS) and Feature Extraction (FE). Fig. 1 shows a graphic representation about how these two approaches work.

On the one hand, Feature Extraction approaches reduce the number of characteristics by the combination (either linear or not) of the input space features [3]. For instance, a Feature Extraction technique in Deep Learning is the so-called deep features, which are the data representation that can be obtained if we remove the last Neural Network (NN) layer. Thus, these techniques are able to create a new feature set, which is often more compact and with higher discrimination capacity. This is the most used technique in image analysis, signal processing and information retrieval [4, 5, 6].

On the other hand, Feature Selection approaches achieve the dimensionality reduction by removing either irrelevant and redundant features [7]. Due to the fact that these techniques are able to preserve the original features, it is an useful approach specially in applications where these attributes are essential to both understanding the model and for the knowledge inferring [8, 9, 10].

Often, FS techniques are divided in three big groups: filters, that work independently of the inductive model (can be viewed as a data pre-processing step); and wrappers and embedded methods, which measure the feature relevance according to a classifiers’ performance. However, the idea is quite similar: having any given dataset, the task is to select which are the features that contains the most discriminative information, in a global way. However, we aim to create an algorithm that, instead of using dataset-level information, takes sample-level information to build a FS model.

For example: suppose we have a medical dataset, and we aim to predict if a patient is prone to have cancer. Classic Feature Selection algorithms select which are the most important features that help the classifier to make a good prediction. On the contrary, we propose to infer, for any specific patient, which are the features the classifier is using to predict a certain output, building a FS algorithm by using this information. We believe this approach can keep all Classic Feature Selection advantages, while adding a new powerful tool: the ability to provide customized feature relevance information for each sample.

The most important work in model explainability is the LIME algorithm [11]. The idea is to apply small perturbations to the input, establishing the importance of each feature in the model’s decision. Although this is a black box method that can be used in any given classifier model, it has two main drawbacks: 1) it is slow, as it requires too much time to evaluate each sample (up to 10 minutos in the ImageNet dataset, as the authors have mentioned in their work); and 2) it cannot help the model to improve its explainability, as it is designed to be used after the model is properly trained.

In this work we propose to solve this problem by introducing a novel Feature Selection algorithm, called Saliency-based Feature Selection (SFS). Our proposal consists in using techniques that are able to provide personalized information for any given example. Once this information is collected, a Feature Selection algorithm is created by including those features that contain a higher discrimination coefficient. As at present, scalability is also an important requirement in Machine Learning algorithms, our algorithm must fulfill also another specific requisite: it must be able to work under Big Data environments.

The rest of the paper is organized as follows: section 2 will explain the personalized information algorithm we are going to use; section 3 will provide the metrics used to compute our saliency; section 4 will propose our novel SFS algorithm; section 5 will show an ablation study regarding our approach; finally, section 6 will show some experimental results over public datasets, and section 7 will offer some conclusions and future work.

2 Personalized Information Algorithm

There are only a few research works that address the problem of FS in Big Data environments. In [12, 8] the authors use Deep Bayesian Networks (Deep BN) to select the most relevant features. Although Deep BN can be used in Big Data, they only provide results in small datasets (either few examples or few features). In [13], the authors claimed to use a deep feature selection technique to reduce the input space in short-term wind forecasting models. However, their approach uses Recursive Feature Elimination (RFE) which requires to exponentially train several different models, making it unable to be used with a high number of features.

Perhaps the most interesting work is presented in [14]. It is called Deep Feature Selection (DFS), and consists on a Elastic Net variant, that can be introduced as an extra layer into any Neural Network (NN). However, the authors claim that the number of layers has to be reduced so as to have the method properly working. Thus, it is not suitable for working with Convolutional Neural Networks (CNN) models. This method introduces a mask in the input data, adding a $l_{1}$ and $l_{2}$-regularization to that mask, in the same way the Elastic Net is defined. Furthermore, Elastic Net penalties are also applied to the hidden layers.

However, as early mentioned, none of these approaches is able to, for any given example, give personalized information. To address this issue, we propose to use a well-known computer vision technique, which is called saliency.

2.1 Saliency

Saliency is a technique that was first developed in computer vision problems. Its goal is to evaluate the degree of quality for each pixel within an image [15]. Often, NNs are seen like a black box where, given any input and any desired output, it is possible to obtain an accurate prediction that is somehow close to what we should expect. However, NNs do not provide any kind of transparent explanation about how the system reaches the predicted solution. Saliency was created with the purpose of seeing what is happening inside the NN. Nowadays, there are two different approximations to calculate this saliency: supervised or semi-supervised.

The first works used a semi-supervised approach [15, 16]. The idea is simple: having any given trained NN for classification, and any given image, a back-propagation routine is used to detect which are the pixels that most influenced the desired output.

Supervised approaches are more recent [17, 18], and they are also trained in a different way. In this case, the NN has the same input and output size. Furthermore, we also know, for any given image in the training set, which are the most relevant features. For instance, if we are detecting a cat, the important features are the pixels in the image that belong to the cat. The model is trained so that the predicted output matches our previous segmentation of important features. This is the reason why these networks are also called Semantic Segmentation Networks. They are also called Attention Models [19, 20] whenever a Recurrent Neural Network is used at the end to evaluate the quality of the feature.

Supervised techniques achieve better results than semi-supervised, but they have a major drawback: it is necessary to know, a priori, which are the most relevant features for each instance, in order to successfully train the model. In the case of an image dataset it is easy, but it is not always possible to obtain this information in other environments, like in the case of DNA microarrays, for instance. Furthermore, the supervised techniques can only be used in classification problems. They are not suitable for other approaches, like regression.

Because of that, we propose to use a semi-supervised saliency technique.

2.2 Our Approach

For our model, we are going to use a generalization of the idea proposed in [15]. Let $\mathbf{X}\in\mathcal{R}^{N\times R}$ be our input data, with $N$ being the number of samples and $R$ the number of features; and let $\mathbf{\tilde{Y}}=f(\mathbf{X};\mathbf{\Theta})\in\mathcal{R}^{N\times C}$ be our classification model (just for the purposes of explaining the model; later we will show how our approach can also be applied in regression problems). It does not matter which type of model we are employing, as long as it can be trained by using a loss minimization function (NN, CNN, SVM, …). $C$ is the number of different classes to evaluate, and $\mathbf{\Theta}$ are the classifier weights, which are adjusted during the training procedure.

For the purpose of explanining the idea, we can assume that $f(\mathbf{X};\mathbf{\Theta})$ is the result of applying the softmax function to a one layer model ($\Theta\in\mathcal{R}^{C\times R}$). Thus, we have that

[TABLE]

where

[TABLE]

$y_{c}^{(i)}$ is the probability that the instance $i$ belongs to the class $c$. $\theta_{c}$ is the $c$-th column of $\Theta$.

To train this model, a loss function $\ell(\mathbf{\Theta};f,\mathbf{X},\mathbf{Y})$ is minimized, where $\mathbf{Y}\in\mathcal{R}^{N\times C}$ is the class one-hot encoding. Since we are using the softmax function as our output, our minimization function is the categorical cross-entropy, defined as

[TABLE]

2.2.1 Classic Saliency

In order to know which the features most contribute to activating the class $c$, the solution proposed in [15] is to evaluate the gradient of $y_{c}^{(i)}$ with respect to the input. To put it in mathematical terms,

[TABLE]

To give an intuition behind the idea, this gradient indicates how we should modify the input instance in order to maximize its belonging to class $c$. This technique, however, only works in classification problems. Thus, we need to create a generalization of this method for our purposes.

2.2.2 Generalization Approach

Instead of applying a gradient function for each class $c$, our idea is similar to the one that is used to update the weights during the training procedure. As the loss function usually gives higher gradients whenever there is a strong training misclassification, we propose the definition of a Gain Function ($g(\mathbf{\Theta};f,\mathbf{X},\mathbf{Y})$), that will bring high gradients whenever a sample is correctly classified. Thus, our Saliency Function $\sigma$ will be defined as

[TABLE]

where $\mathbf{\tilde{y}}=f(\mathbf{X};\mathbf{\Theta})$ is our model’s predicted output for the instance $i$.

Below we will show how to create this Gain Function $g$, depending on the loss function we are trying to minimize.

3 Building the Gain Function

As early mentioned, our aim is to develop a saliency system that can work with multiple types of problems. To that end, we are going to explain how to create our Gain Function $g$ in three different scenarios: one for regression and two for classification (NNs and SVMs).

3.1 Regression

First, we will introduce the Regression Gain Function, as it is the most intuitive one. As simplification, we assume our model is trained by using the Mean Square Error Loss (MSE):

[TABLE]

where $\mathbf{\tilde{Y}}=f(\mathbf{X};\mathbf{\Theta})$ is our model’s predicted output. It does not matter if you choose a different loss function, because all regression losses have the same structure: the loss is [math] if the prediction is perfect, and the loss increases as the prediction moves away from the expected result.

On the contrary, our Gain Function must behave in the opposite way: it must have high values whenever the prediction is good, and values close to zero with poor predictions. Thus, our solution is to use the inverse of the MSE loss function:

[TABLE]

where $\alpha$ is a multiplication factor and $\epsilon>0$ is a factor to avoid division by zero. By default, we set $\alpha=1$ and $\epsilon=10^{-3}$.

3.2 Classification

Although Eq. 7 might also be used as the Gain Function in classification problems, we found it not very suitable. The reason is its behavior when a total misclassification ($\tilde{y}^{(i)}_{c}=0$ and $y^{(i)}_{c}=1$, for instance) occurs. Recall that we are using the saliency function to build a feature selection algorithm. Thus, we do not want to receive any information when a total misclassification occurs. That is, our Gain Function should return [math] values when this happens. Unfortunately, the Gain Function provided in Eq. 7 does not satisfy this requirement.

Consequently, we have developed two different gain functions for two different classification losses: cross-entropy and hinge loss, the ones most commonly used.

3.2.1 Cross-entropy Gain Function

The cross-entropy loss (Eq. 3) is the most common used loss function when dealing with NN for Classification, including CNNs. Since our Gain Function should operate in the opposite way, as discussed above, our proposed solution is

[TABLE]

where

[TABLE]

in order to avoid a zero-logarithm. $\alpha$ and $\epsilon$ have the same behavior exposed in Eq. 7. This function will ensure that no saliency is obtained whenever a total misclassification occurs $(\log(1-0)=\log(1)=0)$. Note that we do not want to kill the gradient when clipping the value in Eq. 9. Thus, the gradient should remain unaltered ($\nabla\hat{y}^{(i)}_{c}=1$).

Note that this idea is somehow similar to the one proposed in [15]. It only differs in one point: while the method proposed in [15] decomposes the last layer network to obtain the saliency, we applied our model directly to a Gain Function, which makes it suitable to use it in other machine learning approaches, as regression. Another advantage of our approach is that it returns close-to-zero gradients whenever there is a misclasification. In this sense, our algorithm is able to indicate that there are no relevant features in a misclassification. On the contrary, the saliency defined in [15] does not take into account this crucial information.

3.2.2 Hinge Gain Function

The hinge loss is often used to train SVMs. In a multiclass problem, it can be defined as

[TABLE]

that is, our correct class prediction will have values higher than $1$, whereas values lower than $-1$ will be obtained for the incorrect classes, as desired.

We may note that the function does not have gradient when the values are higher than $1$ in the correct class (same with $-1$ in the incorrect ones). Thus, a predicted output with value of $2$ should have the same information as a predicted value of $2000$. Thus, we modified the Gain Function as follows:

[TABLE]

where

[TABLE]

Again, we do not kill the gradient after clipping ($\nabla\breve{y}^{(i)}_{c}=1$).

4 Saliency-based Feature Selection

Our approach is named as Saliency-based Feature Selection, or SFS. It is a ranker-based feature selection method, that is, it returns an ordered vector of all features, based on their importance. In Algorithm 1 we show its schema.

The proposed algorithm only contains three hyper-parameters: $\epsilon\geq 1$, a stopping criteria variable; $1\geq\gamma>0$, which controls the number of alive features that are kept in the next iteration; and $reps$, which controls the number of times a model is trained, in order to avoid overfitting.

We start by training the model $f$ with all the features in the feature set. Then, we train the model and we compute the saliency. After that, we sum up and sort all features, obtaining the feature ranking $\mathbf{r}$. Then, we discard the least relevant features, and we repeat the operation until the stopping criteria is satisfied.

The way to compute the saliency differs depending on whether we are dealing with a classification or a regression problem. In the case of the classification issue, the procedure is explained in Algorithm 2. Basically, we compute and normalize the saliency for each class. Alter that, we sum up all features. In case of a regression problem, we just sum up all the saliency scores, as described in Algorithm 3.

The complexity of this algorithm is variable, as it completely depends on the $\gamma$ parameter. In the best scenario, when $\gamma=0$, the complexity is lineal in the number of instances ($\mathcal{O}(N)$), whereas in the worst scenario $\gamma\approx 1$, the complexity also depends on the number of variables ($\mathcal{O}(RN)$), as we barely remove one feature in each loop.

5 Ablation Study

In this section we will show how parameters $\gamma$ and reps affect the behavior or our algorithm. Furthermore, we will show how decoupling the model to obtain the feature ranking from the model that is used to classify can affect our approach. In order to accomplish that, we first introduce the datasets that will be used to test our methodology.

5.1 Datasets

5.1.1 NIPS 2003 Feature Selection Challenge

We have selected the 5 datasets that were proposed in the NIPS 2003 Feature Selection challenge111http://clopinet.com/isabelle/Projects/NIPS2003/. There are 5 synthetic datasets that were designed with the only purpose of measuring the quality of feature selection algorithms for classification. The specific characteristics of each dataset are shown in Table 1. Although each dataset only contains two classes, they are challenging because of these factors: few training examples (Arcene), unbalanced data (Dorothea) or low relevant features (Madelon).

5.1.2 MNIST, FASHION-MNIST, CIFAR-10 and CIFAR-100

Due to their small number of examples, NIPS 2003 FS challenge datasets are not suitable to test the behavior of our approach in Big Data scenarios. To overcome this issue, we have included four classic computer vision datasets:

• •

MNIST It is the most classic computer vision classification challenge [21]. It contains more than 50.000 handwritten digits (10 classes) stored in grayscale $28\times 28$ images.

• •

FASHION-MNIST It is a dataset of Zalando’s article images [22]. It contains 60.000 images (10 classes) stored in grayscale $28\times 28$ resolution.

• •

CIFAR-10 and CIFAR-100 They were proposed in [23]. Each one contains more than 50.000 tiny RGB images ($32\times 32\times 3$) of different objects (car, truck, plane, …). The first one contains images belonging to 10 different labels, whereas the other contains images from 100 different classes.

5.1.3 Regression

As early mentioned, and different from classic information-based FS algorithms [24], our model is able to perform FS in regression problems. To test it we have used two different Big Data datasets:

• •

Relative location of CT slices on axial axis222https://archive.ics.uci.edu/ml/datasets/Relative+location+of+CT+slices+on+axial+axis. Originally published in [25], its aim is to discover the relative location of the image on the axial axis. It contains 53500 CT images that belong to 74 different patients. Each image is reduced to two different histograms, resulting in 385 total features.

• •

Enery Molecule Dataset333https://www.kaggle.com/burakhmmtgl/energy-molecule. Originally published in [26], the dataset contains the ground state energies of 16,242 molecules, each with 1275 features. The aim of this dataset is to use Machine Learning techniques to quickly compute the atomization energy, as the simulations needed to compute it require a high computational time.

5.2 Testing the effects of the algorithm´s parameters

5.2.1 Effect of ‘reps’ parameter

As it is well-known, it is almost impossible to train the same machine learning model more than once, expecting to achieve the very same exact result each time. Aspects like random initialization in model’s weights, or random permutations in the training set cause the model output to be similar but not entirely exact each time the model is trained. Thus, our first aim was to evaluate how the number of train repetitions can affect the output. Our objective was to check if it will be necessary to, at each step, train the model more than once, computing the saliency ranking as the mean average of all repetitions.

We have trained our algorithm using the NIPS 2003 FS Challenge datasets, fixing $\gamma=0$, and trying different number of repetitions. We have trained a 3-layer Fully-Connected NN ($150$, $100$ and $50$ nodes per layer, respectively). Batch Normalization (BN) [27] and $\text{{ReLU}}(x)=max(0,x)$ activation are used. Softmax function is used as output, and Eq. 3 is used as loss function.

We have included an l2 weight decay regularization with factor $0.001$. We have trained the model for 100 epochs, using Adam [28] as optimizer. To deal with unbalanced data, we have replicated the number of examples until we achieved a true balance between classes. All models were created using Keras Framework444https://keras.io/, along with TensorFlow [29] as back-end.

Fig. 2 shows the results obtained. It can be seen that the number of repetitions used affects accuracy in most datasets, and that this effect depends also in the number of features. We have conducted a Friedman test, resulting in there is no significant difference between the models when the number of repetitions is higher than 2, except in the Madelon dataset, where there is a low number of relevant features. We also found significant differences between these models against the model with just one repetition. Thus, we recommend to use a number of repetitions higher than 2 to achieve a better performance.

5.2.2 Effect of $\gamma$ parameter

In order to carry out the experiments to analyze the case of the behavior of the$\gamma$ parameter, we have fixed $\mbox{reps}=1$, while rest of parameters were the same as the in previous subsection. Fig. 3 shows that the accuracy of our algorithm improves as the $\gamma$ value increases. This occurs because when removing some features, some redundancy is also eliminated, causing some other features to become more important. Although increasing the $\gamma$ value helps our algorithm to achieve better scores, the Friedman tests we have conducted suggest there are no significant differences when we select $\gamma\geq 0.3$, except in the Madelon dataset. Furthermore, a Wilcoxon test suggest there is a significant difference between $\gamma=0$ and $\gamma=0.75$ models over the datasets Arcene, Madelon and Gisette. However, we found no significant difference over the Dexter and Dorothea datasets. In these datasets, we presume the high features/samples ratio are causing this effect.

5.2.3 Effect of decoupling ranker and classifier

Our Feature Selection algorithm is an embedded model, as both selection and classification/regression tasks can be performed at the same time. An embedded system selects features that achieve good results in the same model that is used to perform either the classification or the regression task. However, this might led us to select features that are only valid for the machine learning model we are using to obtain the ranking.

This fact arises one question: How good is our selection?. Consequently, we tested our proposal separating the problem into two different tasks: ranking and classifying. By using different models for each task, we will test how dependant is our ranking with respect to the model that was used to obtain it. To this end, we used the four kernel implementations provided by the sklearn’s SVC dataset [30], fixing the parameters $C=1$, $degree=3$ and $coef0=1$. Our algorithm meta-parameters were fixed to $\gamma=0.975$ and $reps=1$. We do not need to use more repetitions, as the SVM SMO-training algorithm [31] is very stable. Table 2 shows the obtained results over the NIPS datasets.

Two main answers arise:

Only in $35\%$ of cases the best result is provided by using the same algorithm to obtain both the ranking and the classification. Since we are using 4 different models, this percentage is close to random. 2. 2.

In three different datasets (Arcene, Dorothea and Gisette) the best ranker is the same for three different classifiers. This suggests the selected features are not substantially affected by the type of classifier that is selected. Thus, a good ranker model performs well no matter which classifier we use afterwards.

Thus, it seems clear that it is more important to have a good classifier when selecting the features, rather than trying to use the same model for either ranking and classifying.

5.2.4 Effect of overfitting in the FS step

As we have seen in the previous subsection, a good classifier selection is crucial in order to obtain a solid feature selection, and thus we also would like to evaluate how the overfitting of a model might affect the selection of features. In this case, we have only used the Arcene dataset to illustrate the scenario, as it provides very meaningful results.

In the previous subsection, it can be seen that the best results for Arcene dataset were achieved using SVM-RBF for both ranking and classifying, with $C=1.5$ as meta-parameter. Thus, our experiment consists in, using the same classifier configuration, check how the result varies as we modify the $C$ parameter during the FS step. Our algorithm meta-parameters were fixed to their minimum values ($\gamma=0$ and $reps=1$).

Fig. 4 shows the obtained results. The best results are achieved when using the same $C$ value that was employed also during the classification step. It can be seen that using a lower value does not substantially affect the result. However, when introducing overfitting (high $C$ values), the quality of the result is drastically reduced.

Thus, we strongly recommend, during the FS step, the use of classifiers that are able to prevent overfitting, as this is a problem that could introduce noise in the FS ranking system.

6 A comparative study of our proposal with other FS methods

In order to test the adequacy of our proposal, we have compared it against the most representative feature selection methods available nowadays, namely: (a) both Sklearn’s LASSO and Elastic Net implementations [32, 33]; (b) the MIM revision developed in [34], also implemented in the Sklearn package; (c) the ReliefF algorithm [35] implemented in the Skrebate package [36]; and (d) the DFS algorithm [14], using our own Keras implementation. In this last case, and in order to make a fair comparison, we have used the DFS mask values as the feature ranking. This approach was not mentioned in the original paper mentioned above [14], as they just train the model with the new mask and constraint (like LASSO and Elastic Net behavior). However, experimental results show that this approach achieves better results when using our variation. The code with our implementations is available in GitHub555https://github.com/braisCB/SFS.

6.1 NIPS 2003 Feature Selection Challenge

As the number of instances is relatively low in all datasets, we have followed the same approach explained in the ablation study described in the previous section, that is, we have used four SVM variants to test the performance of every algorithm. In the case of the DFS algorithm, we have used the three-layer NN that was also explained in the previous section as the feature selection algorithm. We also have tried the same network with our proposed approach.

6.1.1 Arcene Dataset

Table 3 shows the obtained results. Overall, the best results are achieved when using the polynomial kernel. However, for all the other tested algorithms but our approach, these are achieved when using practically almost all features (in the most favourable case, 81,6% of the total features). Differently, our algorithm is able to achieve the best same results in accuracy as the polynomial kernel, but when using a RBF kernel and only $19$ features, that is, the 4,87% of the whole features of the dataset.

6.1.2 Dexter Dataset

The problem found with the polynomial kernel in the Arcene dataset appears also in Dexter dataset, although this time with the Linear kernel, as we can see in Table 4. Again, our algorithm outperforms the other techniques, independently of the SVM kernel selected. This time our proposal not only reduces considerably the number of features used (100% for the best result in other methods, while ours use 7,1%), but also improves slightly the accuracy result obtained.

6.1.3 Dorothea Dataset

Table 5 shows that the best result is achieved when using the DFS algorithm with an RBF kernel. However, our approach is able to achieve the same accuracy, both with Polynomial and RBF kernels, but requiring more features (57 for the DFS algorithm and 283 for our proposal, respectively 0,057% and 0,3% regarding the complete set of features).

6.1.4 Gisette Dataset

Table 6 shows that our algorithm achieves the highest score when using the Neural Network as ranker, although DFS obtains similar accuracy with fewer features (130 for DFS versus 291 for our approach, respectively 2,6% and 5,8% of the total set of features). Compared with either MIM or ReliefF, our algorithm is able to systematically select fewer features.

6.1.5 Madelon Dataset

The ReliefF algorithm achieves the best score, as Table 7 shows. However, we may note that our algorithm is also able to achieve the same score, but using more features (1,8% and 5% of the total number of features).

To sum up, our algorithm is able to achieve the same, or even improve slightly, the accuracy results compared with those obtained by the state-of-the art algorithms for all NIPS datasets. Regarding the number of features, our proposal behaves extraordinarily in some datasets, in which the reduction of the features needed is remarkable, while in other cases in which the existing methods accomplish an important reduction our approach needs approximately the double.

6.2 Regression

We conducted two experiments to show the behavior of our SFS algorithm in a regression problem. As early mentioned, we have used both the Relative location of CT slices on axial axis and the Energy Molecule dataset . To perform the experiments, we selected the same approach presented in Section 5.2.1: a 3-layer NN ($150$, $100$ and $50$ nodes, respectively), along with BN and ReLU activation function. Both differ only in the output (now it is just one node) and the loss function (MSE). A 5-fold cross-validation was performed.

Table 8 shows the obtained results for the Relative location of CT slices on axial axis dataset. We compare our approach with the DFS algorithm, using an input mask with $l_{1}=5\cdot 10^{-4}$ as weight penalty. Additionally, we also consider using our SFS method adding the DFS mask at the input. We refer to that configuration as SFS$+$DFS. We have fixed the parameter reps to $2$ in all experiments. The results show that SFS and the combination SFS$+$DFS, both with a 3-layer CNN, achieved the best results. For the SFS alone, 192 features, that is the whole set of features were needed, while for the combination of DFS with our approach, only 96 (50%) were needed for an almost equal accuracy.

Using the same configuration, Table 9 shows the obtained results for the Energy Molecule dataset. The results show that SFS with $\gamma=0.9$ is able to achieve almost the best score by just using 127 features (the 10% of the whole dataset). Either SFS or SFS$+$DFS achieve the best scores in all different configurations.

6.3 Results of the approach in Big Data scenarios

One of the main advantages of our method is that it is possible to use it in Big Data environments, as it was developed to be used in state-of-the-art architectures like Convolutional Neural Networks. To that end, we have tested the behavior of our approach over four different datasets using the Wide Residual Network WRN-16-4 [37] as classifier.

As ranking method, we have tried two different configurations: the previously mentioned WRN-16-4 and a standard 3-layer CNN. It consists in two convolutional layers with 16 and 32 channels, respectively. After each convolution, a $2\times 2$ Max-Pooling Layer. Finally, two Fully Connected layers are used: the first one contains 1024 nodes, and the last one has the size of the number of labels in the dataset ($100$ for CIFAR-100, $10$ for the others). Both Batch Normalization (DN) [27] and ReLU activation function are applied right after all hidden layers, and the Softmax function is applied to the output. The Adam optimizer [28] is used to train the model.

In both networks we use the categorical cross-entropy as loss function, together with an $l_{2}=5\cdot 10^{-4}$ weight penalty.

We have conducted experiments in four well-known image databases described in sections above: MNIST, Fashion-MNIST, CIFAR-10 and CIFAR-100.

6.3.1 MNIST

No data augmentation techniques were used to train the models. The 3-layer CNN was trained for 40 epochs, while the WRN-16-4 required 80 epochs. Table 10 shows the obtained results. As it can be observed, our approach achieved the best scores balancing accuracy and number of features used. It is worth mentioning that the accuracy of the classifier highly improves when using a high $\gamma$ value. In the next section we will show our intuition behind this effect.

6.3.2 Fashion-MNIST

In this case, as data augmentation we used random horizontal flips, along with horizontal and vertical random shifts (up to $4$ pixels). As this dataset is more complex than MNIST, we increased the number of training epochs to $80$ in the case of the 3-layer CNN, and $130$ when using the WRN-16-4. Table 11 shows the obtained results. Overall, our SFS approach with $\gamma=0.9$ achieved the best scores.

6.3.3 CIFAR-10

We used for this case the same data augmentation and training configuration that was presented in the previous Fashion-MNIST dataset, but increasing the random shifts up to $5$ pixels. Table 12 shows the obtained results. The use of DFS helps the model to achieve better results whenever the number of kept features is low. On the contrary, a high $\gamma$ value is useful with a higher number of features (more than $25\%$ of the total features).

6.3.4 CIFAR-100

We used the same training configuration as in the previous CIFAR-10. Table 13 shows the obtained results. In this dataset our results are not clearly better than DFS. We believe that the overfitting problem is causing this results (although the training accuracy is close to 100%, the test accuracy never reaches 70%).

6.4 Checking Classifier’s Reliability

As we mentioned before, to our knowledge this is the first Feature Selection algorithm, besides Tree-based techniques, that is able to provide the feature importance for each sample. The main advantage of our proposal is that we can provide an explanation about the classifier’s decision. However, we would like to focus on a different scenario: checking the classifier’s reliability.

If we take a look at the MNIST results (Table 10), we can see that the best results are achieved when using as ranker the 3-layer CNN instead of the WRN-16-4 model. This result was not expected because the latter is a better classifier than the former. Thus, our intuition in this effect was reduced to two points:

Max-pooling: We think the main disadvantage of our algorithm is that it does not manage feature correlation, unless the training algorithm does it. In the case of a CNN model, local correlations can be detected by using Pooling layers. Contrary to the WRN-16-4 classifier, the 3-layer CNN contains 2 Max-pooling layers, which could help our algorithm to achieve better scores. 2. 2.

Over-fitting: During the training step, the training set accuracy always reaches a perfect score when using the WRN-16-4. This could led our algorithm to a bad generalization behavior, as explained in the ablation study section (see Fig. 4).

Thus, we conclude that the quality of the classifier highly affects the behavior of our algorithm. And the MNIST results also show that test accuracy is not a good metric to determine how suitable is a classifier to be used as ranker in our algorithm.

For this reason, we carried on a different experiment, taking advantage of saliency’s properties. As we have previously defined, our saliency function is the gradient of the gain function with respect to the input. Thus, it measures how we have to modify our input to increase the probability of belonging to the desired class. So, we decided to use our saliency function to do exactly the opposite. The idea is to answer this simple question:

• •

How much do I have to change a sample to change the classifier’s output?

This topic, called Adversarial Examples gained a lot of relevance during these years. Several techniques were created in order to increase the reliability of a classifier. For instance, Fast Gradient Sign Method (FGSM) [38], Fast Gradient Sign Method (FGSM) and its variants [39], and Projected Gradient Descent (PGD) [40] use the Saliency gradient to evaluate how much an input sample must be modified to cheat the classifier. However, we want to extend these works, in order to evaluate how reliable is a classifier. We do not focus on how much an image has to be modified, but to see if the generated image can also cheat a human eye.

To answer this question, we have conducted an experiment, that is shown in Table 14. Given an original image and a trained classifier, we use the saliency output to make perturbations in this image, in order to cheat the classifier and obtain wrong predictions with a high confidence (higher than $95\%$). The first row shows the perturbations needed when using a WRN-16-4 model, which achieves a $99.69\%$ accuracy on the testing set. We can see that the resulting images have no substantial visual difference with respect to the original one. It looks like only random noise was added to the original image. To put it in different words, ten different images that look extremely similar are able to achieve completely different predictions in our WRN-16-4 model.

As we conclude that introducing white noise can easily fool our WRN-16-4 model, we re-trained it again, but this time introducing some random Gaussian noise in the training images. Apparently, the quality of the classifier decreases, as we only obtain a $99.37\%$ accuracy on the testing set. However,when we take a look at the images (Table 14, second row), we can see that the perturbations look completely different from the original image, and that it should be easy to establish the classifier prediction by just looking at the input. Therefore, we conclude that, although the second model achieves a lower accuracy in the training set, it is a more reliable model.

This is a very interesting effect that could led to potential implications in the training algorithm. As we can easily obtain images that are able to fool our model, we can make adjustments to our training set so we can improve the reliability of our model. This is a very important factor in topics like decision support systems, as we can base our conclusion in a more solid explanation.

It could also have potential implications in medical analysis, as, having any given sample, we can show to doctors which is our conclusion, and how the input should be modified to modify the prediction. Together with a dictionary of potential treatments and their effect to our input parameters, it could also led to treatment recommendations.

7 Conclusion

In this paper we have presented a novel Feature Selection ranking approach, called Saliency-based Feature Selection (SFS). Contrary to classic Feature Selection approaches, our algorithm is able to rank the importance of each feature at an instance-level, rather than as a whole dataset. Besides this advantage, experimental results in challenging datasets show that our algorithm is able to achieve state-of-the-art results under different configurations, making it suitable to be used in any king of problem, either is it classification or regression. Contrary to classic Information-based Feature Selection techniques, the reduced complexity of our SFS algorithm (it can be computed simultaneously with the classification or regression training) allows it to be used in high-dimension datasets.

As future research, we aim to use our SFS technique to define a metric that evaluates a model’s degree of robustness, that is, to measure how hard is to fool either a classifier or a regression architecture, instead of just using visual cues. We also want to test our adversarial images as part of a explainable model in a real scenario like medical information.

8 Acknowledgements

This research has been financially supported in part by the Spanish Ministerio de Economía y Competitividad (research project TIN2015-65069-C2-1-R), and by the Xunta de Galicia (research projects ED431C 2018/34 and Centro singular de investigación de Galicia, accreditation 2016-2019), and the European Union (European Regional Development Fund - ERDF). We also gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research. Brais Cancela acknowledges the support of the Xunta de Galicia under its postdoctoral program.