Study and Observation of the Variation of Accuracies of KNN, SVM, LMNN, ENN Algorithms on Eleven Different Datasets from UCI Machine Learning Repository

TL;DR

This study evaluates the accuracy variations of KNN, SVM, LMNN, and ENN algorithms across eleven UCI datasets to understand their performance differences and relationships with data characteristics.

Contribution

It implements and compares four supervised learning algorithms on multiple datasets to analyze accuracy variations and performance relationships.

Findings

Accuracy varies significantly across datasets for each algorithm.

Performance differences highlight the influence of data characteristics.

Insights into algorithm suitability for different data types.

Abstract

Machine learning qualifies computers to assimilate with data, without being solely programmed [1, 2]. Machine learning can be classified as supervised and unsupervised learning. In supervised learning, computers learn an objective that portrays an input to an output hinged on training input-output pairs [3]. Most efficient and widely used supervised learning algorithms are K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Large Margin Nearest Neighbor (LMNN), and Extended Nearest Neighbor (ENN). The main contribution of this paper is to implement these elegant learning algorithms on eleven different datasets from the UCI machine learning repository to observe the variation of accuracies for each of the algorithms on all datasets. Analyzing the accuracy of the algorithms will give us a brief idea about the relationship of the machine learning algorithms and the data dimensionality. All the algorithms are developed in Matlab. Upon such accuracy observation, the comparison can be built among KNN, SVM, LMNN, and ENN regarding their performances on each dataset.