On the Effect of Ruleset Tuning and Data Imbalance on Explainable Network Security Alert Classifications: a Case-Study on DeepCASE

TL;DR

This paper examines how label imbalance affects the performance and explainability of DeepCASE, a state-of-the-art network intrusion alert classifier, and shows that tuning detection rules can improve results.

Contribution

It demonstrates that rule tuning reduces data imbalance and enhances both classification accuracy and explanation quality in automated SOC alert classification.

Findings

Label imbalance impacts classification performance.

Tuning detection rules reduces imbalance.

Improved data quality benefits automation.

Abstract

Automation in Security Operations Centers (SOCs) plays a prominent role in alert classification and incident escalation. However, automated methods must be robust in the presence of imbalanced input data, which can negatively affect performance. Additionally, automated methods should make explainable decisions. In this work, we evaluate the effect of label imbalance on the classification of network intrusion alerts. As our use-case we employ DeepCASE, the state-of-the-art method for automated alert classification. We show that label imbalance impacts both classification performance and correctness of the classification explanations offered by DeepCASE. We conclude tuning the detection rules used in SOCs can significantly reduce imbalance and may benefit the performance and explainability offered by alert post-processing methods such as DeepCASE. Therefore, our findings suggest that traditional methods to improve the quality of input data can benefit automation.