Towards a Standard Feature Set for Network Intrusion Detection System Datasets

TL;DR

This paper proposes and evaluates standardized NetFlow-based feature sets for network intrusion detection datasets to improve comparability and generalization of machine learning models across different network scenarios.

Contribution

It introduces two standardized NetFlow-based feature sets for NIDS datasets, addressing the lack of a common feature standard in the field.

Findings

The 12-feature set provides a lightweight option for NIDS.

The 43-feature set offers a more comprehensive profile for intrusion detection.

Standardized features facilitate better comparison and evaluation of ML-based NIDS.

Abstract

Network Intrusion Detection Systems (NIDSs) are important tools for the protection of computer networks against increasingly frequent and sophisticated cyber attacks. Recently, a lot of research effort has been dedicated to the development of Machine Learning (ML) based NIDSs. As in any ML-based application, the availability of high-quality datasets is critical for the training and evaluation of ML-based NIDS. One of the key problems with the currently available datasets is the lack of a standard feature set. The use of a unique and proprietary set of features for each of the publicly available datasets makes it virtually impossible to compare the performance of ML-based traffic classifiers on different datasets, and hence to evaluate the ability of these systems to generalise across different network scenarios. To address that limitation, this paper proposes and evaluates standard NIDS feature sets based on the NetFlow network meta-data collection protocol and system. We evaluate and compare two NetFlow-based feature set variants, a version with 12 features, and another one with 43 features.