Deep Learning for Malicious Flow Detection

TL;DR

This paper introduces a novel deep learning approach using a Tree-Shaped Deep Neural Network and a Quantity Dependent Backpropagation algorithm to improve malicious flow detection in imbalanced cybersecurity datasets, demonstrating superior performance and real-time capabilities.

Contribution

The paper presents a new end-to-end trainable neural network architecture and a class disparity-aware training algorithm for effective imbalanced data learning in cybersecurity.

Findings

Outperforms state-of-the-art methods on imbalanced datasets.

Demonstrates real-time detection feasibility.

Shows generalization capability through zero-shot learning.

Abstract

Cyber security has grown up to be a hot issue in recent years. How to identify potential malware becomes a challenging task. To tackle this challenge, we adopt deep learning approaches and perform flow detection on real data. However, real data often encounters an issue of imbalanced data distribution which will lead to a gradient dilution issue. When training a neural network, this problem will not only result in a bias toward the majority class but show the inability to learn from the minority classes. In this paper, we propose an end-to-end trainable Tree-Shaped Deep Neural Network (TSDNN) which classifies the data in a layer-wise manner. To better learn from the minority classes, we propose a Quantity Dependent Backpropagation (QDBP) algorithm which incorporates the knowledge of the disparity between classes. We evaluate our method on an imbalanced data set. Experimental result demonstrates that our approach outperforms the state-of-the-art methods and justifies that the proposed method is able to overcome the difficulty of imbalanced learning. We also conduct a partial flow experiment which shows the feasibility of real-time detection and a zero-shot learning experiment which justifies the generalization capability of deep learning in cyber security.