Burstiness of Intrusion Detection Process: Empirical Evidence and a Modeling Approach

TL;DR

This paper investigates the bursty and memory-rich nature of intrusion detection events in large networks, using empirical data and hidden Markov models to understand their underlying stochastic processes.

Contribution

It provides the first empirical analysis of intrusion detection burstiness and introduces a hidden Markov model approach to characterize these processes.

Findings

Intrusion detection events exhibit significant burstiness and strong memory.

A hidden Markov model effectively captures the bursty behavior of network incidents.

Understanding burstiness can improve risk quantification and defense strategies.

Abstract

We analyze sets of intrusion detection records observed on the networks of several large, nonresidential organizations protected by a form of intrusion detection and prevention service. Our analyses reveal that the process of intrusion detection in these networks exhibits a significant degree of burstiness as well as strong memory, with burstiness and memory properties that are comparable to those of natural processes driven by threshold effects, but different from bursty human activities. We explore time-series models of these observable network security incidents based on partially observed data using a hidden Markov model with restricted hidden states, which we fit using Markov Chain Monte Carlo techniques. We examine the output of the fitted model with respect to its statistical properties and demonstrate that the model adequately accounts for intrinsic "bursting" within observed network incidents as a result of alternation between two or more stochastic processes. While our analysis does not lead directly to new detection capabilities, the practical implications of gaining better understanding of the observed burstiness are significant, and include opportunities for quantifying a network's risks and defensive efforts.