A Review of Various Datasets for Machine Learning Algorithm-Based Intrusion Detection System: Advances and Challenges

TL;DR

This paper reviews various datasets and machine learning techniques used in intrusion detection systems, analyzing their effectiveness, challenges, and future research directions in cybersecurity.

Contribution

It provides a comprehensive review and analysis of existing datasets and ML methods for IDS, highlighting challenges and potential improvements.

Findings

Datasets like KDDCUP'99, NSL-KDD, and CICIDS-2017 are commonly used.

Machine learning classifiers such as SVM, Random Forest, and Neural Networks are effective in IDS.

Challenges include dataset limitations and the need for more robust detection methods.

Abstract

IDS aims to protect computer networks from security threats by detecting, notifying, and taking appropriate action to prevent illegal access and protect confidential information. As the globe becomes increasingly dependent on technology and automated processes, ensuring secured systems, applications, and networks has become one of the most significant problems of this era. The global web and digital technology have significantly accelerated the evolution of the modern world, necessitating the use of telecommunications and data transfer platforms. Researchers are enhancing the effectiveness of IDS by incorporating popular datasets into machine learning algorithms. IDS, equipped with machine learning classifiers, enhances security attack detection accuracy by identifying normal or abnormal network traffic. This paper explores the methods of capturing and reviewing intrusion detection systems (IDS) and evaluates the challenges existing datasets face. A deluge of research on machine learning (ML) and deep learning (DL) architecture-based intrusion detection techniques has been conducted in the past ten years on various cybersecurity datasets, including KDDCUP'99, NSL-KDD, UNSW-NB15, CICIDS-2017, and CSE-CIC-IDS2018. We conducted a literature review and presented an in-depth analysis of various intrusion detection methods that use SVM, KNN, DT, LR, NB, RF, XGBOOST, Adaboost, and ANN. We provide an overview of each technique, explaining the role of the classifiers and algorithms used. A detailed tabular analysis highlights the datasets used, classifiers employed, attacks detected, evaluation metrics, and conclusions drawn. This article offers a thorough review for future IDS research.