Enhancing Automata Learning with Statistical Machine Learning: A Network Security Case Study

TL;DR

This paper introduces a novel approach combining interpretable machine learning with automata learning to improve network intrusion detection system verification, reducing model complexity and increasing accuracy.

Contribution

It presents a new method for automata learning from numeric network data by using ML-based data abstraction, enhancing model efficiency and verification capabilities.

Findings

67.5% reduction in states and transitions

28% improvement in accuracy

Enables verification and exploration of system behaviors

Abstract

Intrusion detection systems are crucial for network security. Verification of these systems is complicated by various factors, including the heterogeneity of network platforms and the continuously changing landscape of cyber threats. In this paper, we use automata learning to derive state machines from network-traffic data with the objective of supporting behavioural verification of intrusion detection systems. The most innovative aspect of our work is addressing the inability to directly apply existing automata learning techniques to network-traffic data due to the numeric nature of such data. Specifically, we use interpretable machine learning (ML) to partition numeric ranges into intervals that strongly correlate with a system's decisions regarding intrusion detection. These intervals are subsequently used to abstract numeric ranges before automata learning. We apply our ML-enhanced automata learning approach to a commercial network intrusion detection system developed by our industry partner, RabbitRun Technologies. Our approach results in an average 67.5% reduction in the number of states and transitions of the learned state machines, while achieving an average 28% improvement in accuracy compared to using expertise-based numeric data abstraction. Furthermore, the resulting state machines help practitioners in verifying system-level security requirements and exploring previously unknown system behaviours through model checking and temporal query checking. We make our implementation and experimental data available online.