New PCA-based Category Encoder for Cybersecurity and Processing Data in IoT Devices

TL;DR

This paper introduces a PCA-based category encoding method that enhances machine learning performance on high-cardinality categorical data, especially in cybersecurity and IoT contexts, by reducing data dimensionality and computational load.

Contribution

A novel PCA-based encoding approach that converts categorical variables into numerical features with minimal data expansion, improving ML accuracy and efficiency in cybersecurity and IoT applications.

Findings

Achieves highest accuracy and AUC compared to 17 encoders.

Reduces number of features while maintaining performance.

Enhances processing time efficiency for IoT devices.

Abstract

Increasing the cardinality of categorical variables might decrease the overall performance of machine learning (ML) algorithms. This paper presents a novel computational preprocessing method to convert categorical to numerical variables ML algorithms. It uses a supervised binary classifier to extract additional context-related features from the categorical values. Up to two numerical variables per categorical variable are created, depending on the compression achieved by the Principal Component Analysis (PCA). The method requires two hyperparameters: a threshold related to the distribution of categories in the variables and the PCA representativeness. This paper applies the proposed approach to the well-known cybersecurity NSLKDD dataset to select and convert three categorical features to numerical features. After choosing the threshold parameter, we use conditional probabilities to convert the three categorical variables into six new numerical variables. After that, we feed these numerical variables to the PCA algorithm and select the whole or partial numbers of the Principal Components (PCs). Finally, by applying binary classification with ten different classifiers, we measure the performance of the new encoder and compare it with the other 17 well-known category encoders. The new technique achieves the highest performance related to accuracy and Area Under the Curve (AUC) on high cardinality categorical variables. Also, we define the harmonic average metrics to find the best trade-off between train and test performances and prevent underfitting and overfitting. Ultimately, the number of newly created numerical variables is minimal. This data reduction improves computational processing time in Internet of things (IoT) devices in future telecommunication networks.