Performance Evaluation of t-SNE and MDS Dimensionality Reduction Techniques with KNN, ENN and SVM Classifiers

TL;DR

This paper evaluates the effectiveness of t-SNE and MDS dimensionality reduction techniques in improving classification accuracy across nine datasets using KNN, ENN, and SVM classifiers implemented in Matlab.

Contribution

It provides a comparative analysis of t-SNE and MDS in reducing dimensions and enhancing classifier performance on multiple datasets.

Findings

t-SNE and MDS effectively reduce dataset dimensions by half.

Classification accuracy varies with different DR techniques and classifiers.

t-SNE generally outperforms MDS in preserving class separability.

Abstract

The central goal of this paper is to establish two commonly available dimensionality reduction (DR) methods i.e. t-distributed Stochastic Neighbor Embedding (t-SNE) and Multidimensional Scaling (MDS) in Matlab and to observe their application in several datasets. These DR techniques are applied to nine different datasets namely CNAE9, Segmentation, Seeds, Pima Indians diabetes, Parkinsons, Movement Libras, Mammographic Masses, Knowledge, and Ionosphere acquired from UCI machine learning repository. By applying t-SNE and MDS algorithms, each dataset is transformed to the half of its original dimension by eliminating unnecessary features from the datasets. Subsequently, these datasets with reduced dimensions are fed into three supervised classification algorithms for classification. These classification algorithms are K Nearest Neighbors (KNN), Extended Nearest Neighbors (ENN), and Support Vector Machine (SVM). Again, all these algorithms are implemented in Matlab. The training and test data ratios are maintained as ninety percent: ten percent for each dataset. Upon accuracy observation, the efficiency for every dimensionality technique with availed classification algorithms is analyzed and the performance of each classifier is evaluated.