Host-Based Network Intrusion Detection via Feature Flattening and Two-stage Collaborative Classifier

TL;DR

This paper proposes a hybrid host-based and network-based intrusion detection system using feature flattening and a two-stage classifier, significantly improving detection accuracy for various attack types across multiple datasets.

Contribution

It introduces a novel feature flattening technique and a two-stage collaborative classifier that enhances intrusion detection performance by combining host and network data.

Findings

Improved macro average F1 score by 8.1% on CICIDS 2018 dataset.

Enhanced detection of specific attack classes like DoS-LOIC-UDP by 30.7%.

Generalizes well across multiple datasets.

Abstract

Network Intrusion Detection Systems (NIDS) have been extensively investigated by monitoring real network traffic and analyzing suspicious activities. However, there are limitations in detecting specific types of attacks with NIDS, such as Advanced Persistent Threats (APT). Additionally, NIDS is restricted in observing complete traffic information due to encrypted traffic or a lack of authority. To address these limitations, a Host-based Intrusion Detection system (HIDS) evaluates resources in the host, including logs, files, and folders, to identify APT attacks that routinely inject malicious files into victimized nodes. In this study, a hybrid network intrusion detection system that combines NIDS and HIDS is proposed to improve intrusion detection performance. The feature flattening technique is applied to flatten two-dimensional host-based features into one-dimensional vectors, which can be directly used by traditional Machine Learning (ML) models. A two-stage collaborative classifier is introduced that deploys two levels of ML algorithms to identify network intrusions. In the first stage, a binary classifier is used to detect benign samples. All detected attack types undergo a multi-class classifier to reduce the complexity of the original problem and improve the overall detection performance. The proposed method is shown to generalize across two well-known datasets, CICIDS 2018 and NDSec-1. Performance of XGBoost, which represents conventional ML, is evaluated. Combining host and network features enhances attack detection performance (macro average F1 score) by 8.1% under the CICIDS 2018 dataset and 3.7% under the NDSec-1 dataset. Meanwhile, the two-stage collaborative classifier improves detection performance for most single classes, especially for DoS-LOIC-UDP and DoS-SlowHTTPTest, with improvements of 30.7% and 84.3%, respectively, when compared with the traditional ML XGBoost.