Efficient Feature Selection of Power Quality Events using Two Dimensional (2D) Particle Swarms

TL;DR

This paper introduces a novel 2D particle swarm-based feature selection method that explicitly considers subset size, improving the robustness and effectiveness of feature selection for power quality event classification.

Contribution

The paper proposes a new 2D learning framework for feature selection that incorporates subset cardinality, outperforming existing methods in power quality event classification.

Findings

The 2D learning approach outperforms six other feature selection methods.

Selected features are more robust under noise.

The method effectively classifies 14 classes of PQ events.

Abstract

A novel two-dimensional (2D) learning framework has been proposed to address the feature selection problem in Power Quality (PQ) events. Unlike the existing feature selection approaches, the proposed 2D learning explicitly incorporates the information about the subset cardinality (i.e., the number of features) as an additional learning dimension to effectively guide the search process. The efficacy of this approach has been demonstrated considering fourteen distinct classes of PQ events which conform to the IEEE Standard 1159. The search performance of the 2D learning approach has been compared to the other six well-known feature selection wrappers by considering two induction algorithms: Naive Bayes (NB) and k-Nearest Neighbors (k-NN). Further, the robustness of the selected/reduced feature subsets has been investigated considering seven different levels of noise. The results of this investigation convincingly demonstrate that the proposed 2D learning can identify significantly better and robust feature subsets for PQ events.

Figures

Equations

$$J({\mathcal{X}}^{\star })=\underset{\mathcal{X}\subset U,{\xi }_{\mathcal{X}}<n}{\max }J(\mathcal{X})$$

$${\beta }_i$$

$${\mathcal{V}}_i=\left[\begin{array}{ccc}{p}_{11}^i& \dots & p_{1n}^i\\ p_{21}^i& \dots & p_{2n}^i\end{array}\right]$$

$${\mathcal{V}}_i$$

$$\text{where, }{\mathcal{V}}_{gi}$$

$${\mathcal{V}}_{li}$$

$${\Delta }_i=\{\begin{array}{ll}+{\delta }_i,& if\frac{f_i^t}{f_i^{t-1}}<1\\ -{\delta }_i,& otherwise\end{array}\text{where, }{\delta }_i=1-\frac{f_i^t}{max(F^t)}$$

$$\ell =\{u_1,u_2,\dots u_n,v_k\},v_k\in V$$

$$J(\mathcal{X})$$

$$\text{where, }{e}_f$$

$$PI(\%)$$

$$\Xi (\%)$$

$$\text{overall score}=\sum _{k=1}^{40}(\frac{{\xi }_k}{n})\times J({\mathcal{X}}_k)$$

$$\Theta (\mathcal{X})$$

$$\text{where, }{\eta }_f$$

$$\theta (\mathcal{X})=\Theta (\mathcal{X})-\Theta (U)$$

$${\beta }_i$$

$${\mathcal{V}}_i=\left[\begin{array}{ccccc}0.82& 2.53& 2.22& 0.28& 0.95\\ 1.61& 1.88& 0.80& 1.33& 2.88\end{array}\right]$$

$$\frac{F_s}{2^{D+1}}\le f\le \frac{F_s}{2^D}$$

Full text

††journal: Applied Soft Computing

1 Introduction

Over the past two decades, the landscape of the energy market has been going through a significant transformation due to the increasing share of non-linear loads. The breakthrough progress in the semiconductor technology has enabled a wide-scale deployment of the power electronic converters, adjustable speed drives and consumer electronics. Furthermore, the power electronic converters are the key components in the grid interface of the renewable generation. In this scenario, one of the major challenge faced by the utilities is the deterioration in Power Quality (PQ). Since the success of the remedial action is critically dependent on the nature of the PQ event, the identification of a PQ event is vital in addressing the poor PQ. This study, therefore, focuses on identification of PQ events.

In essence, any departure from the ideal sinusoid can be considered as a PQ event. Various PQ events exist in the modern grid which differ in terms of spectral content, magnitude and duration. Based on these ‘traits’, the IEEE standard 1159 characterizes PQ events of distinct nature [1]. Arguably, the earliest attempt to identify the PQ events could be traced back to the works of Santoso et al. [2, 3], Angrisani et al. [4] and Gaouda et al. [5], in which the feature extraction through multi-resolution analysis was proposed. Subsequently, over the years, the PQ event identification problem has transformed into a pattern recognition problem wherein a pattern is formed by various features extracted from the voltage measurements and a label corresponding to PQ event, i.e., $i^{th}$ pattern, ‘$\ell_{i}$’, is given by, $\ell_{i}=\{u_{1}^{i},u_{2}^{i},\dots u_{n}^{i},v_{i}\}$, where, ‘$v_{i}$’ and $\{u_{1}^{i},u_{2}^{i},\dots u_{n}^{i}\}$ respectively represent the output label and the features extracted from the $i^{th}$ PQ event. The patterns extracted from the PQ events are subsequently used to induce a classifier through a suitable induction algorithm. Most of the existing PQ event identification approaches are based on this framework [6, 7]. Note that the utility/efficacy of the extracted features is usually not evaluated in most of the existing PQ identification approaches [8, 9, 10, 11, 12], which often leads to the inclusion of irrelevant, redundant and noisy features.

Despite the significant research efforts dedicated to the machine learning and pattern recognition, the feature extraction process is still, in major part, dependent on the expert knowledge. Since it is not trivial to estimate the required number of features/attributes beforehand, a large number of features are usually extracted in order to identify a pattern. This often leads to the inclusion of irrelevant and/or redundant features which often lead to increased storage requirement, slower processing times and reduced generalization capability of the classifier. The objective of the feature selection is to overcome these problems by the removal of irrelevant/redundant features. Note that the feature selection is one of the fundamental problems of the machine learning and it has been topic of active research since last six decades [13, 14, 15, 16, 17]. Over the years, several feature selection approaches have been developed with a proven ability to significantly reduce the number of features while maintaining/improving classification performance in many applications [16, 17].

Despite these advantages, the feature selection has not received enough attention in PQ identification except for a few existing research [18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30]. The choice of the feature selection method varies among the existing research which include Sequential Search [19, 25, 26, 27], Genetic Algorithm (GA) [18, 19, 21, 22, 28], Simulated Annealing (SA) [21, 22], Binary Particle Swarm Optimization (BPSO) [28], Fully Informed Particle Swarm (FIPS) [20], Artificial Bee Colony (ABC) [29], k-means apriori algorithm [23] and rough sets [24]. The common drawback of the sequential search method such as Sequential Forward Search (SFS) and Sequential Backward Search (SBS) is the so-called ‘nesting effect’, i.e., once the feature is included/excluded from the subset, it cannot be removed/added. The floating sequential search approaches such as Plus-l Take Away-r (PTA) and Sequential Forward Floating Search (SFFS) can prevent nesting effect. However, these are still local search, as the over-emphasis on a single feature neglects the correlation among features. Furthermore, all sequential search methods require specification of the reduced subset size a priori. Given that the size of the optimal feature subset is not known, the sequential search essentially addresses only half of the feature selection problem. The meta-heuristic search methods such as GA, SA and ABC could overcome these limitations. However, in [18, 22, 29], these approaches have not been used to full potential; as the size of the feature subset is fixed a priori. Such practice is usually not recommended, since fixing the subset size to a particular value, say ‘$d$’, will reduce feature search space from $2^{n}$ to $\binom{n}{d}$ which may exclude the optimal feature subset from the search. The filter based feature selection approaches have been employed in [25, 26, 24] which usually involves a trade-off in the classification performance as the nature of the induction algorithm is not considered. This is further, corroborated by the results of the earlier investigation [28] which suggest that the indirect performance indicators used in the filter approach do not necessarily translate into equivalent classification performance. In [20], Probabilistic neural network-based Feature Selection (PFS) has been proposed which employs a combination of Fully Informed Particle Swarm and Adaptive Probabilistic Neural Network to evaluate the efficacy of a feature subset. The major drawback of this approach is the assumption of the monotonic relation between the feature subset size and classification performance. This assumption can only be satisfied by an ideal Bayes classifier. In practice, the classifiers do not follow the monotonicity property [31, 32, 33, 34]. In addition, PFS is prone to the nesting effect, as it does not have any mechanism to reconsider the discarded features.

Further, in most of the existing research, the deterministic feature selection approaches have been included in the comparative analysis whereas it is known that meta-heuristic search could yield enhanced search performance [17]. However, among the meta-heuristic search, so far only the performance of GA, SA and BPSO have been evaluated [18, 21, 22, 28]. In this scenario, the following questions arise:

Amongst various meta-heuristic approaches (such as GA, BPSO, ACO and others) which approach would yield the minimum feature subset without compromising the classification performance? 2. 2.

Whether the classification performance obtained from the reduced feature subset is robust against various levels of measurement noise?

The main aim of this study is to address these concerns through a comprehensive investigation. For this purpose, a comparative analysis of seven different feature selection wrappers is carried out using fourteen distinct class of PQ events and two different induction algorithms. Further, the robustness of the reduced subsets against the measurement noise is evaluated through seven different levels of zero-mean Gaussian white noise.

The other objective of this study is to investigate the efficacy of the new Two-Dimensional feature selection algorithm on the PQ events. The Two-Dimensional (2D) learning algorithm has recently been developed by the authors as a generalized feature selection wrapper [35]. The core idea of the 2D-learning approach is to embed the information about the number of features (referred to as ‘cardinality’) into the learning process of particle swarms. This distinctive quality of the 2D learning approach has been shown to be more effective in identifying the compact feature subsets with the improved classification performance for several benchmark machine learning datasets. This, therefore, has been the motivation behind the application of 2D learning to the PQ event identification problem. In particular, the present study focuses on the practical issues encountered in the feature selection of PQ events and therefore significantly differs from [35] in the following aspects:

• •

A well-balanced, comprehensive and diverse PQ event dataset is built. The dataset includes both simulated and experimental PQ events which conform to the IEEE Std. 1159. Further, a relatively large variation of PQ events is considered in comparison to the existing research on PQ identification [20, 11, 36, 9, 30, 8]. For example, the PQ events include magnitude variation in the range of $[0,\ 4]\ pu$, event duration from $0.5$ to $30$ cycles, frequency variation in the range of $[0,\ 10]\ kHz$, and the harmonics of the order $(6k\pm 1),\ k=\{1,2,3,\dots\}$. Thus, most of the PQ events expected in the conventional power distribution grid are well represented in the investigated dataset.

• •

The robustness of the selected/reduced feature subsets against the measurement noise is defined and evaluated under various levels of zero-mean Gaussian white noise.

• •

The Two-Dimensional (2D) learning algorithm is better formalized and explained through an illustrative example from the perspective of PQ event identification.

• •

The efficacy of 2D learning algorithm is demonstrated by the comparative evaluation on six established feature selection algorithms: GA, ACO, BPSO, CBPSO, ChBPSO and SFFS.

• •

A rigorous analysis is carried out to determine the statistical significance of the results. For instance, multiple non-parametric statistical comparisons are carried out using the Friedman test and the Hommel’s post-hoc procedure to compare the search performance of the algorithms. In addition, the performance of algorithms under various levels of noise is compared using the Contrast Estimation based on medians.

Thus, in this study, most of the operating scenarios expected in the utility distribution grid have been accommodated. Further, several key issues related to the feature selection have been investigated.

The rest of the article is organized as follows: Section 2 provides a brief overview of the feature selection approaches. The 2D-learning approach has been briefly discussed in Section 3. The investigation framework of this study is provided in Section 4. The results of the comparative evaluation and the robustness test are shown in Section 5, followed by the discussions in Section 6 and the conclusions in Section 7.

2 Brief Overview of Feature Selection Methods

Since the main objective of this work is to select the relevant feature subset for the PQ event identification, it is pertinent to discuss the existing feature selection methods briefly.

The feature selection problem essentially begins by considering a dataset having ‘$n$’ input features, $U=\{u_{1},\dots u_{n}\}$ and ‘$m$’ output class, $V=\{v_{1},\dots v_{m}\}$. For this dataset, the task of the induced classifier is to determine the output label ‘$v_{k}$’ ($v_{k}\in V$) corresponding to the input features. The objective of the feature selection is to identify the subset of features, ‘$\mathcal{X}^{\star}$’, ($\mathcal{X}^{\star}\subset U,\;\xi_{\mathcal{X}^{\star}}<n$) through which this task can be accomplished with similar or improved classification performance:

[TABLE]

where, ‘$J(\cdotp)$’ is the criterion function which represents the classification performance of the feature subset and ‘$\xi$’ denotes the cardinality or number of features in the subset. The search for an optimal solution of the feature selection requires the evaluation of $2^{n}$ subsets. Hence the exhaustive search is intractable even for the moderate size datasets [37]. Note that the selection of a relevant feature subset from the available features is essential for the better classification performance and the generalization capability of the classifier [14].

Over past six decades, while the fundamental objective of the feature selection problem has not changed (i.e., identify a subset of relevant features), the machine learning algorithms have evolved significantly with time to meet the requirements of different applications. For example, in many applications, the class ‘labels’ may not be available completely or partially [16, 38]. Such scenarios require unsupervised/semi-supervised machine learning approach. A distinct feature selection strategy is required for each machine learning approach such as supervised or unsupervised learning [16, 38, 39, 40]. In this study, we focus on the feature selection approaches for supervised learning; as the nature of PQ events have been well characterized by the IEEE Std. 1159 [1]. Hence, a comprehensive PQ event dataset has been developed through a combination of simulation and practical field measurements for the supervised learning.

Most of the existing feature selection methods for supervised learning can be distinguished by their approach (e.g., wrapper or filter) to evaluate the criterion function, $J(\cdotp)$ [14, 15, 33]. The wrapper approach is straightforward where a classifier is induced for each feature subset under consideration and the resulting classification accuracy is used as $J(\cdotp)$. On the contrary, in the filter methods, $J(\cdotp)$ is estimated using statistical or information theory without inducing a classifier. Note that the search landscape of the feature selection problem is conjointly defined both by the dataset and the induction algorithm, i.e., each induction algorithm has specific traits. Hence, the optimal feature subset for particular induction algorithm may not be optimal for the other, as will be shown by the results of this study (Section 5.1).

The choice of feature selection method involves a trade-off in either speed or accuracy; the wrappers are more precise whereas filters are comparatively faster. The size of the feature set plays a major role in the selection of either approach. For the smaller to medium size feature set ($n<1000$) wrappers are more appropriate. For the larger dataset, the computational burden of the wrapper may be infeasible. Hence filters are preferred in such a scenario. Usually, the PQ event datasets range in small to medium size ($n<500$), thus in this work, the focus is on the wrapper approach.

Earlier approaches of feature selection, such as sequential search methods [13, 41, 42, 43] and branch and bound methods [44, 45] are ‘deterministic’ in nature, i.e., for a given dataset, they give the same solution over independent runs. The core idea behind most of these approaches is to evaluate the utility/relevance of a feature by evaluating its discrimination capability over the output classes. However, due to over-emphasis on an individual feature, the correlation among features is neglected. In addition, most of the deterministic approaches require a priori selection of subset cardinality, ‘$d$’. Consequently, the search space reduces from $2^{n}$ to $\binom{n}{d}$. Further, several deterministic approaches [44, 45] assume a monotonic criterion function, $J(\cdotp)$, which is often impractical.

Meta-heuristic search methods such as Genetic Algorithm (GA), Ant Colony Optimization (ACO), Tabu Search (TS) and Particle Swarm Optimization (PSO) have been applied to the feature selection problem to address the shortcomings of the deterministic approaches [17]. Unlike deterministic methods, most of the meta-heuristic search methods operate on feature subsets. Hence, the effects of feature correlation are accounted in the search process. Further, the meta-heuristic search is, in essence, a population-based search with implicit parallelism which allows comparatively better sampling of the search space. For this reason, we focus on the meta-heuristic wrappers in this study.

3 Two-Dimensional (2D) Learning Framework for Particle Swarms

The core idea behind the Two-Dimensional (2D) learning approach has been briefly discussed in the following subsections for the sake of completeness. The detailed discussion about this approach can be found in [35]. Note that, 2D learning is intended to be a generalized learning algorithm for particle swarm based feature selection methods. It is, therefore, possible to adapt most of the existing PSO variants [46] in the continuous domain ($x\in\mathbb{R}$) for the feature selection problems ($x\in\mathbb{N}$) following the 2D-learning. However, the results of the study in [47] indicate that among popular PSO variants, adapted Unified Particle Swarm Optimization (UPSO) [48] performs comparatively better for the problems in the discrete domain. For this reason, in this work, UPSO has been adapted following the 2D-learning approach and referred throughout the manuscript as ‘2D-UPSO’.

3.1 Philosophy of 2D Learning

The search for the optimal feature subset essentially entails the following two decisions: 1) How many features should be included? and 2) Which features should be included? The philosophy of the Two-Dimensional (2D) learning is to explicitly embed the information about subset size (also referred as cardinality) into the search process to effectively address these issues. As the name suggests, the 2D-learning approach extends the learning dimension of a particle swarm to integrate the cardinality information into the search process. Since the cardinality of the subset is selected through an informed decision, only the features with higher selection likelihoods are selected and the redundant features are effectively discarded.

To further understand the philosophy of 2D learning, consider a feature selection problem associated with a dataset having ‘$n$’ number of features. For this dataset, the position of an $i^{th}$ particle, ‘$\beta_{i}$’, is represented as an $n$-dimensional binary string as follows:

[TABLE]

where, the bits with ‘$1$’ indicates that the corresponding feature is selected and the bits with ‘[math]’ indicates otherwise, i.e., if $\beta_{i,m}=1$ then $m^{th}$ feature is selected.

Note that in PSO, the ‘learning’ of a particle is accumulated in its velocity. For this reason, in 2D learning, the dimension of particle velocity is extended, and it is represented by a two-dimensional matrix. The objective here is to store the selection likelihood of feature and cardinality (number of features) in a distinct dimension. For the problem considered here, the velocity of the $i^{th}$ particle, ‘$\mathcal{V}_{i}$’, is represented by a two-dimensional matrix of size ($2\times n$) which is given by,

[TABLE]

The first row of the velocity matrix stores the selection likelihood of the cardinality (i.e., subset size or number of features). For instance, $p_{13}^{i}=2.45$ implies that the likelihood of including a total of $3$ features in the new position of the $i^{th}$ particle is $2.45$. In contrast, the elements in the second row of $\mathcal{V}_{i}$ store the selection likelihood of the corresponding features. For example, $p_{2,3}^{i}=0.75$ indicates that likelihood of including the $3^{d}$ feature in the new position of the $i^{th}$ particle is $0.75$.

Note that in order to update the selection likelihoods of cardinality and features it is crucial to extract the beneficial information from the ‘learning exemplars’ such as ‘personal best’ ($pbest$) and ‘neighborhood best’ ($nbest$). This is accomplished through ‘learning’ process in which a dedicated ‘learning set’ ($\mathcal{L}$) is derived from each exemplar. The learning set is also a $(2\times n)$ matrix, in which the first row corresponds to the cardinality learning and the second row corresponds to the feature learning. In essence, the learning process extracts the following information from the exemplar and encodes it into a two-dimensional binary learning set ‘$\mathcal{L}$’: 1) number of features in the exemplar 2) features that have been included in the exemplar but not in the particle.

This is achieved as follows: Let ‘$\alpha$’ denote the learning exemplar of the $i^{th}$ particle, e.g., ‘$pbest_{i}$’, ‘$nbest_{i}$’ or ‘$gbest$’. Note that the learning exemplar,‘$\alpha$’, is also an $n$-dimensional binary string similar to particle position and essentially encodes a feature subset. Further, let $n$-dimensional vectors,‘$\varphi$’ and ‘$\psi$’, respectively denote the ‘cardinality’ and ‘feature’ learning set. For the $i^{th}$ particle, ‘$\beta_{i}$’, and the corresponding learning exemplar, ‘$\alpha$’, the learning sets are derived following the procedure outlined in Algorithm 1. This procedure is further explained by the illustrative example in A.

3.2 Velocity and Position Update

In this study, the well-known PSO variant, UPSO, has been adapted for the feature selection through the 2D-learning approach. The core idea behind UPSO [48] is to combine the ‘global’ [49] and ‘local’ [50] version of the PSO to achieve the balance between exploration and exploitation of the search landscape. The velocity update rule of UPSO adapted through 2D learning approach (referred as ‘2D-UPSO’) is given by,

[TABLE]

and ‘$u$’ is the unification factor; ‘$\omega$’ is the inertia weight; the parameters $[c_{1},c_{2}]$ are acceleration constants; $[r_{1},r_{2}]\in[0,1]$ are uniform random numbers; ‘$\mathcal{L}_{soc,1}$’ and ‘$\mathcal{L}_{soc,2}$’ are the social learning sets derived respectively from global best ($gbest$) and neighbourhood best ($nbest_{i}$). Similarly, ‘$\mathcal{L}_{i}$’ denotes the learning set derived from the current position of the particle, $\beta_{i}$. The adaptive weight used to control the influence of $\mathcal{L}_{i}$ is denoted as ‘$\Delta_{i}$’ and it is given by,

[TABLE]

where, ‘$f_{i}^{t}$’ is the fitness of the $i^{th}$ particle and ‘$F^{t}$’ is the vector which contains the fitness of the entire swarm at iteration ‘$t$’. For a given particle, the parameter ‘$\delta$’ is adaptive and its value is dependent on the relative performance of the particle with respect to the worst particle of the swarm. As seen in (7), the particle with the minimum fitness will have higher $\delta$, which will lead to the increase in selection likelihood of the corresponding features (included in the particle). Further, as seen in (7), $\delta$ is positive only when the particle leads to improved fitness.

Following the velocity update, the new position of the particle is determined in two steps. In the first step, the cardinality of the new position is determined which is followed by the selection of the features. For this purpose, the selection likelihoods stored in the velocity matrix are used. This procedure is outlined in Algorithm-2 and further explained through an illustrative example in A. The pseudo code for 2D-UPSO is outlined in Algorithm 3.

4 Investigation Framework

In this study, we focus mainly on two aspects of the feature selection in PQ events: 1) selection of an appropriate feature selection method and 2) robustness of the reduced subsets against the measurement noise. For this purpose, this investigation has been carried out in two stages, as shown in Fig. 2. The objective of the first stage is to carry out the comparative analysis of different feature selection approaches. For this purpose, seven different search algorithms (discussed in Section 4.3) are applied as a meta-heuristic wrapper to two induction algorithms (discussed in Section 4.2) and fourteen distinct types of PQ events (discussed in Section 4.1). Note that, the PQ events used for the feature selection purpose do not contain any noise, i.e., Signal-to-Noise Ratio (SNR) $=\infty\ dB$. In the second stage, the robustness of the reduced subsets, obtained from the feature selection, is evaluated. For this purpose, the classification performance of the reduced subsets is evaluated in the presence of $7$ levels of zero mean Gaussian white noise: SNR=$[50,45,40,35,30,25,20]\ dB$.

The following subsections provide more details about the PQ events, Search Algorithms and Induction Algorithms being used in this study.

4.1 PQ Events & Feature Extraction

For the comprehensive study, it is necessary to include a wide variety of PQ events in the investigation. For this purpose, the IEEE Std. 1159 [1] is followed in this study. A total of fourteen distinct PQ events are generated through parametric models given in Table 1 and the experimental setup shown in Fig. 2. These events include several distinct natures of the PQ events, e.g., stationary ($v_{2},v_{7},v_{8}$), non-stationary ($v_{3},v_{4},v_{5},v_{9},\dots v_{14}$), low frequency ($v_{1},v_{2},\dots v_{7}$), high frequency ($v_{8},v_{9},\dots v_{14}$), single ($v_{1},v_{2},\dots v_{9}$) and simultaneous ($v_{10},v_{11}\dots v_{14}$).

The synthetic PQ events were generated in MATLAB® following the parametric models shown in Table 1. The real PQ events are acquired using the experimental PCC shown in Fig. 2, which consist of a harmonic source and a transient source. The harmonic source is being emulated by a three-phase uncontrolled rectifier with a resistive load which generates the harmonics of order $(6k\pm 1),\ k=\{1,2,3,\dots\}$. The transient events are induced by switching of a capacitor bank. Further, the sag events are induced by creating a single line to ground fault. A digital oscilloscope/recorder (HIOKI 8870-20 MEMORY HiCORDER®) is used to capture the real events. For each class of PQ event, $250$ instances are generated which gives a total of $3500$ instances of PQ events. Approximately $90\%$ of these event instances are generated through the parametric models shown in Table 1 and the remaining are induced using the experimental PCC shown in Fig. 2. Each event instance is generated at the fundamental frequency of $50\ Hz$, for the duration of $30$ cycles and sampled at $25\ kHz$. The PQ events include magnitude variation in the range of $[0,4]\ pu$, frequency variation in the range of $[0,10]\ kHz$, harmonics of order $\{5,7,11,13,17,19,23,25,29,31,\dots\}$ and the event duration from $0.5$ to $30$ cycles. In addition, to evaluate the effects of the measurement noise, seven different levels of zero-mean Gaussian white noise have been added to these events. This gives a total of $8$ PQ datasets; each dataset has the same $3500$ events but a different level of the measurement noise, i.e., SNR$=[\infty,50,45,40,35,30,25,20]\ dB$.

For accurate identification of the PQ events, it is crucial to extract information in both temporal and frequency domain. This task can be accomplished through various signal processing techniques such as Stockwell-Transform (ST), Wavelet Packet Transform (WPT), Discrete Wavelet Transform (DWT) [6, 7]. Note that the selection of signal processing technique is primarily dependent on the frequency bandwidth of the signal being investigated. Especially, a judicious selection is essential to accommodate the relatively large frequency bandwidth of PQ events, e.g., from DC offset ($0\ Hz$) to Oscillatory Transients ($5\ MHz$) [1]. In this scenario, among existing signal processing techniques, DWT represents an ideal choice for PQ events as it is computationally more efficient; the complexity of DWT, WPT and ST is respectively $\mathcal{O}(N)$, $\mathcal{O}(NlogN)$, $\mathcal{O}(N^{2}logN)$ ($N$* denotes the number of samples*). Therefore, DWT has been selected as a signal processing technique in this study.

It is well-known that the selection of the base/mother wavelet is dependent on the nature of the application and it is crucial to the performance of wavelet transforms. The detailed investigation on this topic [51] suggest that for the PQ events, the optimum classification performance of the given induction algorithm can be obtained when it is paired with a specific base wavelet. For example, the optimum performance from k-Nearest Neighbor (k-NN) and Naive-Bayes (NB) (which are being used in this study) was obtained with the $6^{th}$ order symlet (‘$sym6$’) [51]. Therefore, in this study, ‘$sym6$’ has been selected as the base wavelet.

Further, each instance of the PQ event is decomposed to the $8^{th}$ level using DWT (the decomposition level has been selected following the rule of thumb given in B). Consequently, the detail coefficients at each level and approximation coefficients at the final level are available. In order to extract meaningful information from the wavelet coefficients, $11$ statistical functions are used (shown in C). Following this procedure, for each instance of PQ event, a total of $99$ ‘features’ ($9$ details/approximations $\times$ $11$ functions) are obtained. Hence, a pattern, ‘$\ell$’, is obtained corresponding to each instance of PQ event, as follows:

[TABLE]

where, ‘$n=99$’ denotes the number of features; $U=\{u_{1},u_{2},\dots u_{n}\}$ denotes the features set and $V=\{v_{1},v_{2},\dots v_{14}\}$ contains the label corresponding to each PQ event.

4.2 Induction Algorithm

The results of our previous investigation [51] suggest that relatively simple induction algorithms are more robust against the measurement noise. In particular, Decision Tree (DT) and Naive-Bayes (NB) [52] were found to be comparatively more robust for PQ events. Since DT has limited inherent feature selection capability, we have selected NB as one of the induction algorithms in this study. Further, it is well-known that the search landscape of the feature selection problem is conjointly defined by the dataset and the induction algorithm [14, 15]. For this reason, in addition to NB, k-Nearest Neighbor (k-NN) [52] is also used as an induction algorithm in this study.

Note that the classification performance of IA is critically dependent on the ‘hyperparameters’ which are used to control the learning process, e.g., ‘number of neighbors’ ($k$) and ‘distance metric’ in k-NN; ‘kernel width’ ($\sigma$) in NB. In this study, the hyperparameters of both IAs have been selected through ‘grid-search’ to maximize the average ten-fold classification accuracy (‘$\Theta$’), as shown in Fig. 3(a) (k-NN) and Fig. 3(b) (NB).

4.3 Compared Feature Selection Algorithms

To evaluate the efficacy of the 2D learning approach, the following existing algorithms are considered: Genetic Algorithm (GA) [31, 53], Ant Colony Optimization (ACO) [54, 55], Binary PSO (BPSO) [56], Catfish BPSO [57] and Chaotic BPSO (chBPSO) [58]. In all these algorithms, the search agent (e.g., a chromosome in GA) encodes a feature subset by the binary string representation given in (2).

In this study, the classical variant of GA is considered, i.e., simple GA with roulette wheel selection, single-point crossover and flip-bit mutation operator, as outlined in [31, 53]. For ACO, the variant proposed in [54, 55] is considered, where the features are represented as nodes on a directed graph. Each node on this graph is linked by two distinct edges to highlight whether a node/feature is selected or not. A feature subset is represented by a path traversed by an ant over these edges. For the given edge, the probability of inclusion in the path is given by the corresponding pheromone intensity. In each iteration, the pheromone intensity is updated based on the positive feedback mechanism. In this study, the pheromone update procedure proposed for ACO in [54] is implemented. Further, since the proposed 2D-UPSO has been developed in particle swarm theory, BPSO and its two variants (CBPSO and chBPSO) are also included in the comparative investigation. The CBPSO [57] retains the learning mechanism of BPSO while introducing the concept of ‘refresh gap’, i.e., a fixed number of worst performing particles are reinitialized if the swarm cannot locate an improved solution for a pre-fixed number of iterations. In chBPSO [58], the velocity update rule of BPSO is updated to control the inertia weight following the chaotic maps. In this study, a logistic map is being used to determine the value of inertia weight in chBPSO.

In addition to the meta-heuristic algorithms, the Sequential Forward Floating Search (SFFS) [42] is also included to compare the search performance of the 2D learning with the existing deterministic search algorithms. The SFFS has been selected for this purpose, as it overcomes the limitations of the other sequential search methods [42], e.g., nesting effects of SFS and SBS; selection of appropriate ‘$(l,r)$’ in plus-l take away-r floating search.

The search parameters of each algorithm are set following the procedures outlined in [31, 54, 56, 57, 58] and are shown in Table 2. All of the compared algorithms are implemented in MATLAB.

4.4 Search Setup

The efficacy of the feature subset can be evaluated by either the filter or the wrapper approach. The selection of either approach requires a trade-off between precision and the computational complexity. In this study, the total number of features is moderate ($n=99$), and therefore it is appropriate to select the wrapper approach due to its precision [14]. In other words, for each subset under consideration, a classifier is induced by the induction algorithm (say NB or k-NN) and the subsequent classification error is used as the criterion function, $J(\cdotp)$.

Further, to remove any bias to the validation data, the mean classification error after 10-fold stratified cross-validation is used as the criterion function, $J(\cdotp)$ [52]. For a given feature subset, $\mathcal{X}$, this is given by,

[TABLE]

Without loss of generality, the feature selection problem is approached as a minimization problem where the objective is to minimize the classification error, $J(\cdotp)$ given by (9). To account for the inherent stochastic nature of the algorithms, $40$ independent runs of each algorithm are executed. Each run is set to terminate after $6000$ Function Evaluations (FEs).

5 Results

As mentioned earlier, in this study we address the following two issues: 1) Which feature selection approach is more effective for PQ events? and 2) How robust are the reduced feature subsets against the measurement noise? For this purpose, the comparative evaluation of seven feature selection wrappers has been carried out using two induction algorithms considering fourteen distinct PQ events, following the procedure outlined in ‘Stage-I’ of Fig. 2. Note that the feature selection has been carried out using only the ‘pure’ PQ events, i.e., with SNR$=\infty\ dB$. The results of this investigation have been discussed in Section 5.1.

The next issue is to evaluate the robustness of the reduced feature subsets which are obtained by the feature selection algorithms. For this purpose, the following seven different levels of zero mean Gaussian white noise are added to the PQ events: SNR$=[50,45,40,35,30,25,20]\ dB$. The framework for this part of the investigation is outlined in ‘Stage-II’ of Fig. 2. The results of this test are discussed in Section 5.2.

5.1 Stage-I : Comparative evaluation of the feature selection approaches

For the purpose of comparative evaluation, $40$ independent runs of each algorithm are recorded. Since the primary objective is to improve the classification performance through the removal of irrelevant/redundant features, the search performance of the compared algorithms is evaluated by two criteria, i.e., classification performance and size of the feature subset (cardinality).

The results obtained after 40 independent runs of each algorithm with k-NN and NB classifier are shown in Table 3 and 4, respectively. The results obtained with 2D learning approach (2D-UPSO) are shown in the last column of Table 3 and 4. The best results obtained among the compared algorithms are shown in boldface.

To compare the classification performance, the average (Mean) and standard deviation (SD) of the criterion function, $J(\cdotp)$, are shown in Table 3 and 4. Similarly, to compare the reduction in subset size, ‘Mean’ and ‘SD’ of subset cardinality is computed over 40 runs for each algorithm as shown in Table 3 and 4. Further, the following two metrics are used to measure the performance improvement obtained by each of the algorithms,

[TABLE]

where, ‘$J(U)$’ is the criterion function with the original feature set ($U$) and ‘$\overline{J(\mathcal{X})}$’ is the average of the criterion function over 40 runs obtained with the reduced feature sets; ‘$\xi_{avg}$’ is the average cardinality over 40 runs and ‘$n$’ is the total number of features, i.e., $n=99$.

The Performance Improvement, $PI(\%)$, metric gives the improvement in the classification performance with respect to the original feature set, ‘$U$’. The second metric, $\Xi(\%)$, shows the percentage reduction in the cardinality with respect to the total number of features, ‘$n$’. A higher positive value of these metrics implies a better search performance. In addition, the following metric is used to evaluate the overall performance of the algorithms:

[TABLE]

where, ‘$\mathcal{X}_{k}$’, ‘$\xi_{k}$’ and ‘$J(\mathcal{X}_{k})$’ are the feature subset, its cardinality and the corresponding classification error at the end of ‘$k^{th}$’ run of the algorithm; ‘$n$’ denotes the total number of features. Note that this metric incorporates the information about both the cardinality and the classification performance. A lower value of this metric indicates that the algorithm could consistently find feature subsets with a lower cardinality and a lower classification error.

From the results shown in Table 3 and 4, it is clear that the 2D learning approach is most efficient as it gives the lowest classification error amongst the compared algorithms. With both the induction algorithms, 2D-UPSO could achieve the highest $PI(\%)$, approximately $87\%$ (with k-NN, Table 3) and $69\%$ (with NB, Table 4). It is interesting to note that in comparison to GA and ACO, all BPSO variants could yield better subsets with both k-NN and NB.

Further, in the present study, the feature selection issue is approached as a single-objective problem where the primary objective is to minimize the classification error. Therefore, any reduction in the subset cardinality is the direct consequence of the ability of the search algorithms to distinguish useful features from the irrelevant/redundant features. Intuitively, the exploitation of cardinality information in the 2D learning is likely to improve the search performance of 2D-UPSO. The results shown in Table 3-4 corroborates this notion. 2D-UPSO could provide the smallest feature subset with the highest reduction in the cardinality,e.g., $78.9\%$ (with k-NN) and $62.6\%$ (with NB).

The overall performance of the compared algorithms is evaluated using the ‘overall-score’ metric (12), which considers both the cardinality, $\xi$, and the classification performance, $J(\cdotp)$, of the feature subsets obtained by the algorithm over 40 runs. As revealed by (12), a lower score indicates the consistent discovery of a subset with fewer features and lower classification error by the algorithm. As seen in Table 3-4, the overall score obtained by 2D-UPSO is the lowest amongst the compared algorithms which indicates the best overall performance.

The results further show a shift in the search landscape with the change of induction algorithms. For example, with k-NN, $PI(\%)$ from the compared algorithms lies in the range of $53\%-87\%$ (Table 3) whereas with NB the performance gain is comparatively lower (in the range of $32\%-69\%$, Table 4). Similar effects are observed in $\Xi(\%)$ as well; its variation with k-NN and NB classifier lie in the range of $52\%-79\%$ and $50\%-62\%$, respectively. These results further underline the need for the wrapper based feature selection approach.

5.1.1 Nonparametric Statistical Evaluation

Due to the stochastic nature of the compared algorithms, further statistical analysis is carried out to determine the significance of the results shown in Table 3 and Table 4. In particular, the objective of this analysis is to determine whether the results (i.e., $J(\cdotp)$ and $\xi$) obtained by 2D-UPSO, are significantly better than the compared algorithms. For this purpose, multiple non-parametric statistical comparisons are carried out following the guidelines in Derrac et al. [59]. The test is carried out in the following two steps:

First, the Friedman Two-way Analysis of Variance by Ranks [60, 59] is applied to determine whether the performance of two or more compared algorithm is significantly different. For this purpose, the results obtained by the algorithms are ranked from ‘$1$’ (best) to ‘$6$’ (worst). Subsequently, the average value of ranks is determined over 40 independent runs. The test statistic and the corresponding p-value are determined following the procedures outlined in [60, 59] and are shown in Table 5. The p-values obtained through the Friedman statistic strongly suggest a significant difference in the performance of the compared algorithms. Further, the average rankings obtained over 40 independent runs establish that 2D-UPSO is the best amongst the compared algorithms.

In the second step, a set of hypotheses is evaluated for multiple comparisons of 2D-UPSO with the other algorithms. Specifically, the evaluation of five interconnected null hypotheses is required to compare the 2D-UPSO with the five compared algorithms, i.e., GA, ACO, BPSO, CBPSO and chBPSO. Each null hypothesis ($H_{0}$) denotes that the algorithm being compared is significantly better than 2D-UPSO. The test statistic, $p-$value and Adjusted p-values (APV) which are required to evaluate these interconnected hypotheses are determined following the procedure outlined in [59, 61]. The Hommel’s post-hoc procedure is employed to derive the APV from the $p-$value. The outcome of the multiple comparisons for $95\%$ confidence interval are shown in Table 6 (for k-NN) and Table 7 (for NB). These results convincingly demonstrate that, among the compared algorithms, 2D-UPSO could obtain feature subsets with significantly lower cardinality ($\xi$) and classification error, $J(\cdotp)$.

5.1.2 Comparative analysis with SFFS

The results of the comparative analysis with SFFS are shown in Table 8. Note that SFFS requires a priori specification of the subset cardinality, $\xi$. Since the cardinality of the optimum feature subset is not known, the cardinality of the best subset found by 2D-UPSO (out of $40$ runs) is determined and denoted as ‘$\xi_{best}$’. For example, the cardinality of the best subset found by 2D-UPSO with k-NN is $10$ and the same with NB is $30$. Hence, $\xi_{best}=10$ (with k-NN) and $\xi_{best}=30$ (with NB). Subsequently, SFFS is applied as a wrapper to both k-NN and NB to find the feature subset with the cardinality equal to ‘$\xi_{best}$’. Given the same cardinality $\xi_{best}$,the objective here is to determine whether the SFFS could identify the feature subset with the comparable or better accuracy than 2D-UPSO. As expected, the outcome of these tests (Table 8) indicates that SFFS could not yield better feature subset with either k-NN or NB.

5.2 Stage-II : Robustness of the Reduced Subsets

To evaluate the robustness of the reduced subsets against measurement noise, seven different levels of noise with SNR$=[50,45,40,35,30,25,20]dB$ have been added to the PQ events. Given that the noise introduced by the measurement chains cannot be a priori estimated, the objective here is to compare the classification performance of the reduced subsets and the original feature set ($U$) at various noise levels.

Note that the feature selection has been carried out using only pure PQ events, i.e. with SNR=$\infty\ dB$. Further, each run of the compared algorithms provides a different feature subset due to the inherent stochastic nature. To ensure the fair comparison, for each algorithm the feature subset with the minimum classification error, $J(\cdotp)$, out of $40$ runs is selected.

For each feature subset under consideration the average classification accuracy after $10$-fold stratified cross-validation, ‘$\Theta(\cdotp)$’, is recorded at each noise level. This is given by,

[TABLE]

‘$\mathcal{X}$’ and ‘$\Theta(\mathcal{X})$’ respectively denote the reduced subset under consideration and the corresponding classification accuracy. Note that, $\Theta(\mathcal{X})=1-J(\mathcal{X})$.

In order to evaluate the ‘robustness’ of a given feature subset, $\mathcal{X}$, its classification accuracy, $\Theta(\mathcal{X})$, is compared with that of the original full feature set, $\Theta(U)$. As the feature selection was carried out using the ‘pure’ dataset (i.e., SNR$=\infty\ dB$), the performance of all reduced subsets is better compared to $U$ at this noise level. The objective here is to investigate whether the reduced subsets could maintain improved performance in the presence of various other levels of noise. Essentially, at each noise level, we are interested in finding out whether $\theta(\mathcal{X})\geq 0$, which is given by,

[TABLE]

The classification accuracy, $\Theta(\cdotp)$ and the performance difference $\theta(\cdotp)$ of the reduced subsets are shown in Table 10 (with k-NN) and Table 10 (with NB). The metric $\theta(\cdotp)$ in (14) indicates the degree of improvement over $\Theta(U)$, i.e., a higher positive value of $\theta(\cdotp)$ is desirable. It is clear that the reduced subset obtained by 2D-UPSO could achieve the highest $\theta(\cdotp)$ with both k-NN (Table 10) and NB (Table 10). By integrating the cardinality information into the search process, 2D-UPSO could find robust and effective feature subsets. These results convincingly demonstrate that with a proper feature selection approach, it is possible to obtain a robust feature subset that can yield enhanced performance even in the presence of various levels of measurement noise.

Further, the results in Tables 10 and 10 clearly show the influence of the induction algorithms on the search landscape. For example, with k-NN, all algorithms could identify robust feature subset, i.e., a positive $\theta(\cdotp)$ at all noise levels, as seen in Table 10. In contrast, with NB, only 2D-UPSO could yield robust feature subset (Table 10). These results further underline the importance of a feature selection approach.

Finally, the statistical significance of the results shown in Table 10 and Table 10 is determined. For this purpose, it is not feasible to apply the multiple non-parametric statistical comparisons (similar to Section 5.1.1), since the number of datasets is relatively small [59]. Therefore, Contrast Estimation based on medians [61, 59] is applied to compare the algorithms. This test essentially estimates a quantitative performance difference over multiple datasets for all possible pairs of algorithms.

The outcomes of this test are shown in Table 12 (with k-NN) and Table 12 (with NB). Note that a higher positive value of the estimator is desirable for this test. For instance, with k-NN, the contrast estimator for ACO is positive with respect to GA ($0.6658$, Table 12) and chBPSO ($0.1142$, Table 12) which indicate that the ACO could yield better subset in comparison to GA and chBPSO. For each of the algorithms, the positive outcomes are shown in boldface in Table 12 and 12. As seen in Table 12 and 12, the estimator for 2D-UPSO is positive for each pairwise comparison, which further highlights enhanced search performance of 2D-UPSO. Furthermore, the shift in the search landscape can indirectly be illustrated by the search behavior of GA; For instance, with k-NN, the estimator values for GA are negative with respect to all the algorithms (Table 12). In contrast, with NB, the positive estimators for GA are obtained with respect to three algorithms (ACO, CBPSO and chBPSO, Table 12).

6 Discussion

The following observations are inferred from the results of the comparative evaluation (Section 5.1) and the robustness evaluation (Section 5.2):

• •

The feature selection wrappers are often criticized for computational complexity. However, as the results of this investigation suggest, wrappers can yield a significant reduction in the feature subset size while improving the classification performance. For instance, the compared algorithms could reduce the original feature set in the range of $52\%-79\%$ (with k-NN, Table 3) and $50\%-63\%$ (with NB, Table 4) while improving the classification performance in the range of $53\%-87\%$ (with k-NN, Table 3) and $32\%-69\%$ (with NB, Table 4). Since the feature selection is performed only once, the wrapper approach is highly recommended for the PQ events.

• •

With an effective search strategy, it is possible to identify a feature subset which is robust against the measurement noise. In this study, the worst case scenario from the perspective of PQ events has been simulated, i.e., the feature selection is carried out using only ‘pure’ PQ events (i.e., SNR=$\infty$ dB) and subsequently the reduced subsets are evaluated under various levels of measurement noise. Nevertheless, under this scenario, 2D-UPSO could identify robust feature subsets with both k-NN and NB.

• •

The results give empirical evidence for the hypothesis that the nature of the induction algorithm does affect the feature selection landscape. For example, all of the compared algorithms could yield robust feature subset with k-NN; however, only 2D-UPSO could identify a robust feature subset with NB. This further underlines the need for a wrapper based feature selection approach.

7 Conclusions

In this study, the issue of feature selection has comprehensively been investigated in the context of PQ event identification. In particular, the search performance of the Two-dimensional learning approach (2D-UPSO) and six other feature selection wrappers has been compared considering fourteen distinct classes of PQ events. Further, the robustness of the reduced feature subsets has been defined and evaluated under seven different levels of measurement noise. The results of the comparative evaluation convincingly demonstrate that 2D-UPSO can identify significantly better and robust feature subsets for PQ events. The key distinctive property of 2D learning is the integration of information about the subset size into the learning framework. This has been shown to lead to a significant improvement in the search performance in comparison to the other well-known algorithms, e.g., GA, ACO and BPSO.

Without loss of generality, this investigation is based on the assumption that the induction algorithm for the PQ event identification is pre-fixed. If this assumption does not hold or a generalized reduced feature subset is desired, then the filter based feature selection approaches are more appropriate. Hence, a detailed comparative investigation of the different filter approaches such as Mutual Information, Minimum Redundancy Maximum Relevance, Correlation based feature selection may prove to be very useful to both the practicing engineers and the PQ researchers. This could be the subject of further research.

Acknowledgement

Faizal Hafiz is thankful to Education New Zealand for supporting this research through the New Zealand International Doctoral Research Scholarship (NZIDRS).

Appendix A Illustrative Example of 2D Learning

Consider a dataset having $5$ number of features ($n=5$). For this dataset, let the position of the $i^{th}$ particle ‘$\beta_{i}$’ and the learning exemplar ‘$\alpha$’ at particular search iteration ‘$t$’ be given by,

[TABLE]

Evaluation of the Learning Sets

The learning sets derived from $\alpha$ and $\beta_{i}$, as per the Algorithm 1, are as follows:

Set cardinality learning sets to null-vector:

$\varphi_{\alpha}=\varphi_{i}=\begin{bmatrix}0&0&0&0&0\end{bmatrix}$ 2. 2.

Cardinality of $\alpha$ and $\beta_{i}$:

$\xi_{\alpha}=\sum\limits_{m=1}^{n}\alpha_{m}=4$ and $\xi_{i}=\sum\limits_{m=1}^{n}\beta_{i,m}=3$. 3. 3.

Set the $\xi^{th}$ bit of $\varphi$ to ‘$1$’:

$\varphi_{\alpha}=\begin{bmatrix}0&0&0&1&0\end{bmatrix}$ and $\varphi_{i}=\begin{bmatrix}0&0&1&0&0\end{bmatrix}$ 4. 4.

Evaluate feature learning sets:

$\psi_{\alpha}=\{\alpha\wedge\overline{\beta}_{i}\}=\begin{bmatrix}0&0&1&0&1\end{bmatrix}$ and $\psi_{i}=\beta_{i}=\begin{bmatrix}1&1&0&1&0\end{bmatrix}$ 5. 5.

Evaluate the final learning sets:

$\mathcal{L}_{\alpha}=\begin{bmatrix}\varphi_{\alpha}\\ \psi_{\alpha}\end{bmatrix}=\begin{bmatrix}0&0&0&1&0\\ 0&0&1&0&1\end{bmatrix}$ and $\mathcal{L}_{i}=\begin{bmatrix}\varphi_{i}\\ \psi_{i}\end{bmatrix}=\begin{bmatrix}0&0&1&0&0\\ 1&1&0&1&0\end{bmatrix}$

Position Update

To understand the position update procedure, assume that the velocity of the $i^{th}$ particle is given by,

[TABLE]

The new position of the $i^{th}$ particle is determined as per the procedure outlined in Algorithm 2:

Set the new position to a null-vector: $\beta_{i}=\begin{bmatrix}0&0&0&0&0\end{bmatrix}$ 2. 2.

Isolate the selection likelihoods of cardinality and features, $\mathcal{V}_{i}=\begin{bmatrix}\rho\\ \sigma\end{bmatrix}$:

$\rho=\begin{bmatrix}0.82&2.53&2.22&0.28&0.95\end{bmatrix}$ and $\sigma=\begin{bmatrix}1.61&1.88&0.80&1.33&2.88\end{bmatrix}$ 3. 3.

Evaluate the accumulative likelihoods:

$\rho_{\Sigma}=\begin{bmatrix}0.82&3.35&5.57&5.85&6.80\end{bmatrix}$ 4. 4.

Let random number, $r\in[0,\rho_{\Sigma,n}]$, be ‘$3.1$’ 5. 5.

Set $\xi=j$ such that $\rho_{\Sigma,j-1}<r<\rho_{\Sigma,j}$:

$\xi_{i}=2$ as $\rho_{\Sigma,1}<r<\rho_{\Sigma,2}$ 6. 6.

Rank the features on the basis of $\sigma$:

$\sigma=\begin{bmatrix}1.61&1.88&0.80&1.33&2.88\end{bmatrix}$ which gives, $\tau=\begin{bmatrix}3&2&5&4&1\end{bmatrix}$ 7. 7.

Select features with a rank less than or equal to $\xi_{i}$:

$\xi_{i}=2$ and $\tau=\begin{bmatrix}3&2&5&4&1\end{bmatrix}$ which gives, $\beta_{i}=\begin{bmatrix}0&1&0&0&1\end{bmatrix}$

Appendix B Determination of DWT Decomposition Levels

The decomposition level for DWT can be determined from the following thumb-rule:

[TABLE]

where, ‘$F_{s}$’ is sampling frequency, ‘$f$’ is the fundamental frequency and ‘$D$’ denotes the required decomposition levels.

Appendix C Statistical Functions used for Feature Extraction

$f_{1}$: $\min\limits_{i=1\dots N_{l}}c_{i}^{k}$, $f_{2}$: $\max\limits_{i=1\dots N_{l}}c_{i}^{k}$, $f_{3}$: Median; $f_{4}$: $\sum\limits_{i=1}^{N_{l}}|c_{i}^{k}|^{2}$, $f_{5}$: $\frac{1}{N_{l}}\sum\limits_{i=1}^{N_{l}}c_{i}^{k}$ , $f_{6}$: $-\sum\limits_{i=1}^{N_{l}}p_{i}\log p_{i}$, $p_{i}=\frac{|c_{i}^{k}|^{2}}{f_{4}}$

$f_{7}$: $\frac{\sum\limits_{i=1}^{N_{l}}({c_{i}^{k}-f_{5}})^{3}}{(N_{l}-1)\;\sigma^{3}}$, $f_{8}$: $\sigma=\sqrt{\frac{\sum\limits_{i=1}^{N_{l}}(c_{i}^{k}-f_{5})^{2}}{(N_{l}-1)}}$, $f_{9}$: $\frac{1}{N_{l}}\sum\limits_{i=1}^{N_{l}}|c_{i}^{k}-f_{5}|$, $f_{10}$: $\frac{\sum\limits_{i=1}^{N_{l}}(c_{i}^{k}-f_{5})^{2}}{(N_{l}-1)}$, $f_{11}$: $\frac{1}{(N_{l}-1)\;\sigma^{4}}\sum\limits_{i=1}^{N_{l}}({c_{i}^{k}-f_{5}})^{4}$

where, $C^{j}=\{c_{1}^{j},c_{2}^{j},\dots c_{N_{l}}^{j}\}$ and ‘$N_{l}$’ denotes number of detail/approximation coefficients at the $j^{th}$ decomposition level.