Data Dimension Reduction makes ML Algorithms efficient

TL;DR

This paper demonstrates that data dimension reduction techniques like PCA and auto-encoders significantly improve the efficiency and accuracy of machine learning algorithms on image datasets such as MNIST and FashionMNIST.

Contribution

It introduces a combined approach of PCA and auto-encoders for pre-processing, showing enhanced performance in supervised and unsupervised learning tasks.

Findings

Massive improvement in accuracy after pre-processing.

Significant reduction in computation time.

Effective application on MNIST and FashionMNIST datasets.

Abstract

Data dimension reduction (DDR) is all about mapping data from high dimensions to low dimensions, various techniques of DDR are being used for image dimension reduction like Random Projections, Principal Component Analysis (PCA), the Variance approach, LSA-Transform, the Combined and Direct approaches, and the New Random Approach. Auto-encoders (AE) are used to learn end-to-end mapping. In this paper, we demonstrate that pre-processing not only speeds up the algorithms but also improves accuracy in both supervised and unsupervised learning. In pre-processing of DDR, first PCA based DDR is used for supervised learning, then we explore AE based DDR for unsupervised learning. In PCA based DDR, we first compare supervised learning algorithms accuracy and time before and after applying PCA. Similarly, in AE based DDR, we compare unsupervised learning algorithm accuracy and time before and after AE representation learning. Supervised learning algorithms including support-vector machines (SVM), Decision Tree with GINI index, Decision Tree with entropy and Stochastic Gradient Descent classifier (SGDC) and unsupervised learning algorithm including K-means clustering, are used for classification purpose. We used two datasets MNIST and FashionMNIST Our experiment shows that there is massive improvement in accuracy and time reduction after pre-processing in both supervised and unsupervised learning.