HCVR: A Hybrid Approach with Correlation-aware Voting Rules for Feature Selection

TL;DR

HCVR is a hybrid, correlation-aware feature selection method that combines non-iterative and iterative filtering, improving classifier performance by effectively eliminating redundant features.

Contribution

This paper introduces HCVR, a novel hybrid feature selection approach that leverages correlation-aware voting rules for improved dimensionality reduction.

Findings

HCVR outperforms traditional feature selection methods on the SPAMBASE dataset.

The method effectively reduces features while maintaining or improving classifier accuracy.

HCVR combines non-iterative and iterative filtering for better feature relevance assessment.

Abstract

In this paper, we propose HCVR (Hybrid approach with Correlation-aware Voting Rules), a lightweight rule-based feature selection method that combines Parameter-to-Parameter (P2P) and Parameter-to-Target (P2T) correlations to eliminate redundant features and retain relevant ones. This method is a hybrid of non-iterative and iterative filtering approaches for dimensionality reduction. It is a greedy method, which works by backward elimination, eliminating possibly multiple features at every step. The rules contribute to voting for features, and a decision to keep or discard is made by majority voting. The rules make use of correlation thresholds between every pair of features, and between features and the target. We provide the results from the application of HCVR to the SPAMBASE dataset. The results showed improvement performance as compared to traditional non-iterative (CFS, mRMR and MI) and iterative (RFE, SFS and Genetic Algorithm) techniques. The effectiveness was assessed based on the performance of different classifiers after applying filtering.