Detecting unknown HTTP-based malicious communication behavior via generated adversarial flows and hierarchical traffic features

TL;DR

This paper introduces HMCD-Model, a detection system for unknown HTTP-based malicious traffic using generated adversarial flows and hierarchical traffic features, achieving high accuracy and better generalization than existing methods.

Contribution

The paper presents a novel detection model combining adversarial flow generation with hierarchical neural network features and provides a large, real-world dataset for training and evaluation.

Findings

Achieves up to 98.66% F1 score on HMCT-2020 dataset.

Outperforms existing methods by over 20% in detection accuracy.

Demonstrates effective detection of unknown malicious HTTP traffic.

Abstract

Malicious communication behavior is the network communication behavior generated by malware (bot-net, spyware, etc.) after victim devices are infected. Experienced adversaries often hide malicious information in HTTP traffic to evade detection. However, related detection methods have inadequate generalization ability because they are usually based on artificial feature engineering and outmoded datasets. In this paper, we propose an HTTP-based Malicious Communication traffic Detection Model (HMCD-Model) based on generated adversarial flows and hierarchical traffic features. HMCD-Model consists of two parts. The first is a generation algorithm based on WGAN-GP to generate HTTP-based malicious communication traffic for data enhancement. The second is a hybrid neural network based on CNN and LSTM to extract hierarchical spatial-temporal features of traffic. In addition, we collect and publish a dataset, HMCT-2020, which consists of large-scale malicious and benign traffic during three years (2018-2020). Taking the data in HMCT-2020(18) as the training set and the data in other datasets as the test set, the experimental results show that the HMCD-Model can effectively detect unknown HTTP-based malicious communication traffic. It can reach F1 = 98.66% in the dataset HMCT-2020(19-20), F1 = 90.69% in the public dataset CIC-IDS-2017, and F1 = 83.66% in the real traffic, which is 20+% higher than other representative methods on average. This validates that HMCD-Model has the ability to discover unknown HTTP-based malicious communication behavior.