Feature Distribution Shift Mitigation with Contrastive Pretraining for Intrusion Detection

TL;DR

This paper introduces SwapCon, a contrastive pretraining approach for network intrusion detection that enhances robustness against feature distribution shifts, outperforming traditional models and demonstrating significant improvements on real datasets.

Contribution

The paper proposes SwapCon, a novel contrastive pretraining method tailored for NID, which effectively mitigates feature distribution shift and improves detection performance.

Findings

Pretraining increases robustness against feature shifts by over 8%.

Proper embedding strategies further enhance pretrained model performance.

SwapCon outperforms XGBoost and KNN models significantly.

Abstract

In recent years, there has been a growing interest in using Machine Learning (ML), especially Deep Learning (DL) to solve Network Intrusion Detection (NID) problems. However, the feature distribution shift problem remains a difficulty, because the change in features' distributions over time negatively impacts the model's performance. As one promising solution, model pretraining has emerged as a novel training paradigm, which brings robustness against feature distribution shift and has proven to be successful in Computer Vision (CV) and Natural Language Processing (NLP). To verify whether this paradigm is beneficial for NID problem, we propose SwapCon, a ML model in the context of NID, which compresses shift-invariant feature information during the pretraining stage and refines during the finetuning stage. We exemplify the evidence of feature distribution shift using the Kyoto2006+ dataset. We demonstrate how pretraining a model with the proper size can increase robustness against feature distribution shifts by over 8%. Moreover, we show how an adequate numerical embedding strategy also enhances the performance of pretrained models. Further experiments show that the proposed SwapCon model also outperforms eXtreme Gradient Boosting (XGBoost) and K-Nearest Neighbor (KNN) based models by a large margin.