A Question of Context: Enhancing Intrusion Detection by Providing Context Information

TL;DR

This paper demonstrates that incorporating contextual information from industrial networks significantly improves the reliability of intrusion detection systems in identifying malicious activities compared to using network data alone.

Contribution

The study introduces a simulation framework and compares context-based and context-less intrusion detection methods, highlighting the benefits of using context information in industrial network security.

Findings

Context-aware detection outperforms context-less methods in identifying attacks.

Industrial networks' uniformity aids in detecting outliers.

Simulation results show improved detection reliability with context information.

Abstract

Due to the fourth industrial revolution, and the resulting increase in interconnectivity, industrial networks are more and more opened to publicly available networks. Apart from the huge benefit in manageability and flexibility, the openness also results in a larger attack surface for malicious adversaries. In comparison to office environments, industrial networks have very high volumes of data. In addition to that, every delay will most likely lead to loss of revenue. Hence, intrusion detection systems for industrial applications have different requirements than office-based intrusion detection systems. On the other hand, industrial networks are able to provide a lot of contextual information due to manufacturing execution systems and enterprise resource planning. Additionally, industrial networks tend to be more uniform, making it easier to determine outliers. In this work, an abstract simulation of industrial network behaviour is created. Malicious actions are introduced into a set of sequences of valid behaviour. Finally, a context-based and context-less intrusion detection system is used to find the attacks. The results are compared and commented. It can be seen that context information can help in identifying malicious actions more reliable than intrusion detection with only one source of information, e.g. the network.

Tables

Table 1. TABLE I: IPCs that are programmed for each individual product

	Product 1	Product 2	Product 3
IPC 1	x		x
IPC 2		x	x
IPC 3	x	x
IPC 4			x
IPC 5	x		x

Table 2. TABLE II: Transition Probablities for the Products

	Product 1	Product 2	Product 3
Product 1	0.6	0.2	0.2
Product 2	0.4	0.6	0
Product 3	0.6	0.2	0.2

Table 3. TABLE III: Starting Probabilities for the Products

Product 1	Product 2	Product 3
0.8	0.1	0.1

Table 4. TABLE IV: Transition Probabilities for the PLCs

	Start	PLC 1	PLC 2	PLC 3	PLC 4	Finish
Start	0	0.9	0.1	0	0	0
PLC 1	0	0	0.9	0.1	0	0
PLC 2	0	0	0	0.6	0.4	0
PLC 3	0	0	0	0	0.9	0.1
PLC 4	0	0	0	0	0	1
Finish	0	0	0	0	0	0

Table 5. TABLE V: Order of Reconfiguring IPCs per Product

	first	second	third	fourth	fifth
Product 1	IPC 1	IPC 2	IPC 3	IPC 4	IPC 5
Product 2	IPC 5	IPC 4	IPC 3	IPC 1	IPC 1
Product 3	IPC 1	IPC 5	IPC 2	IPC 4	IPC 3

Equations

$$\lambda =(S,V,A,B,\pi )$$

$$\begin{array}{r}{F}_1=2\cdot \frac{precision\cdot recall}{precision+recall}\\ precision=\frac{t_p}{t_p+f_p}\\ recall=\frac{t_p}{t_p+f_n}\end{array}$$

$$\begin{array}{r}{p}_{i,j}=P(X_{t+1}=s_j|X_t=s_i)\\ i,j=1,...,m\end{array}$$

Full text

I Introduction

The increasing interconnectivity and the opening of industrial networks to the outside creates the need for security measures. Such measures, e.g. firewall, antivirus software or Intrusion Detection System (IDS), have been used in office Information Technology (IT) for a long time. They are, however, new to industrial applications. Security was not an issue in industry for two reasons [1]: First, Supervisory Control And Data Acquisition (SCADA) systems were physically separated from the internet. And second, each system is unique due to its singular purpose, making it hard for an attacker to exploit it. Both assumptions hold no longer true as the recent and not-so-recent spectacularly show. This lead to the drastic increase in software and network security for industrial applications. Since industrial networks are different in structure and purpose than office networks, the same solutions cannot be transferred readily. Instead, new security solutions have to be developed. Industrial networks are highly specialised, creating unique but very repetitive traffic [2]. Furthermore, industrial networks are connected to Cyber Physical Systems, meaning they are intended to interact with the real world. And lastly, most protocols in the industrial environment do not employ security mechanisms such as authentication and encryption[3, 4], as is standard in the office world.

This work is structured as follows. In section II, the state of the art is described. After that, the use case that is considered in this work is introduced in section III. The corresponding formal model is derived in section IV. In section V, the analysis is conducted and the results are presented. A conclusion is drawn in section VI.

II State of the Art

There are many different approaches for IDSs; Luh et al. categorized them in their survey Semantics-aware detection of targeted attacks into different groups [5]. They identified the main categories “Host-based”, “Network-based” and “Multi-source” with several sub-categories. As we focus our work on industry networks and their components, we will not focus on individual IDSs for specific hosts, but on Multi-source and Network-based approaches. One popular implementation of a Network-based IDS is Snort by Cisco [6], which has grown from a lightweight IDS to a tool that can be used as a full-grown network-based Intrusion Prevention System (IPS). The signature-based approach utilizes user contributed rules to catch different specific instances of network attacks. Another well known network monitoring framework, which can be used as a network IDS, is Bro [7]. Bro is split into layers, where the “Event Engine” performs several integrity checks before it handles the packets sent between senders an recievers that are organised as touples – Bro decides if it should return the whole packet, just the header or nothing at all, depending on the used protocol. The “Policy Script Interpreter”-layer is used to check if the handled packets generated any events that were specified before and then decides if a Bro script should be executed. These scripts can include several tasks like generating new events, logging functions and modifying internal states. There are more popular open source network IDSs like Suricata, which was first released in 2010 by the Open Information Security Foundation (OISF) [8], and Kismet, which aims at wireless networks [9].

According to Jyothsna et al., another method to categorize IDSs is by classifying them into “signature based detection” and “anomaly based detection” [10]. While the first group looks for patterns of known attacks and compares them with the active system, the latter category – the anomaly based approach – builds a model of the normal behaviour and matches this learned behaviour with the running system. While the first approach can’t find novel attacks, the second one needs training on a normal behaving system. According to Thames and Schaefer, the requirements for security aspects differ in many points for CPS and regular IT systems [11]. As we focus primarily on industry networks with heterogeneous participants like Programmable Logic Controllers, Industrial Personal Computers and CPSs with different operating systems, used protocols and various available resources, we have to use a more adjustable approach.

To address these problems we used an approach with Hidden Markov-Models that uses context information. An approach for the usage of Markov chains was published by Ye [12]. In her paper A Markov Chain Model of Temporal Behavior for Anomaly Detection she introduced her technique where the system learns from historic data to distinguish between normal behavior and intrusive activities. Due further developed that approach in [13] to a HMM IDS to learn patterns of Unix processes. Hu et al. introduced an IDS that uses system call based HMMs [14]. They focussed on reducing the amount of submodels required to model the scenario and the impact of pre-processing data on training time. Yolaçan suggested in his approach to enrich the data with context information [15].

The above mentioned IDSs are host-based and do not focus on other available data sources like network traffic. More recent research also show first advances in the field of HMMs in network-based intrusion detection. Chen proposed a classification method for detecting attacks in different attack stages within regular IT networks [16]. Zohrevand used HMMs to develop a system for anomaly detection in water supply SCADA systems showing that their approach outperforms other contestants making HMMs a feasible approach also for industrial networks[17].

III Use Case Description

There are numerous possible attack vectors on industrial applications. As described in section I, each company’s network is different and therefore a wholesome security solution is infeasible. As we described in previous works, there are, however, common attack motivations and attacker types [18]. Some attackers aim at extracting information, so-called “espionage”. Others try to cause harm, in covert operations or in the open, so-called “sabotage”. Not only the goals, but also the means of achieving them are numerous and strongly depend on the use case. Therefore, creating a model that covers all possible attacker scenarios is impracticable. Instead, we focus on a subset of all attacks and describe a special manifestation of sabotage. We consider an attack scenario where an attacker reprograms a PLC. This reprogramming leads to unexpected behaviour of the CPS with the potential to cause harm and damage. In our opinion, there are two scenarios how the attacker is able to reprogram the PLC, and four possible ways to detect this attack. An attacker can either

• •

take over a valid programming device and use it maliciously or

• •

introduce a malicious device to the network.

How she circumvents the perimeter is beyond the scope of this work, there are several related works discussing physical and network security [19]. Furthermore, the given attack can be detected in one of four ways.

Identity

If each programming device and PLC is provided with a cryptographically secure identity, the adversary has to break it. With state of the art authentication, this can be considered impossible. However, most industrial applications rarely use authentication on devices [3]. This detection mechanism only discovers attacks by an attacker that introduced a malicious device. If malicious programming is executed with a valid device, a valid identity is provided.

Statistically

Depending on the way an attacker reprograms devices, statistical deviations from normal behaviour could be observed. This is possible for both ways an attacker can modify the system as described above, but only if the attacker significantly differs from the standard system behaviour. A Dolev-Yao intruder model [20] assumes the attacker possesses knowledge of the system, enabling her to perfectly mimic the standard behaviour.

Payload-based

Deep packet inspection is employed by several IDS to discover content that is capable of causing harm to a system. In industrial applications, a complex model of the production facility is necessary to determine which parameters can cause damage. Even more, a value that is well within safety boundaries can still lead to the production of goods that are low quality. Therefore, it is difficult to determine malicious traffic by payload in industrial applications.

Context-based

Sometimes, information about network traffic alone is not enough to discover attacks. In this case, adding context information can enrich the data source and help to provide a broader view on a system. This principle is employed by Security Information and Event Management (SIEM) systems.

In this work, we assume that no identity information is provided, allowing an attacker to spoof entities in the network. Furthermore, we assume she does not strictly stick to the standard system behaviour, creating statistical deviations that could be detectable. We then compare the statistical analysis with context-based analysis.

IV Formal Model

For analysis, a formal model was created. The formal model was used to generate data, as well as to analyse the data with respect to anomalies induced by attacks. The model consists of three parts:

• •

Products

• •

IPCs

• •

PLCs

The products symbolise the intended output of a fictional production process. Depending on the change of product, several IPCs have to be reconfigured. These IPCs that are reconfigured are a subset of all available IPCs. Each IPC controls four PLCs that are reprogrammed in case of reconfiguration. The order of reprogramming the PLCs follows a probability distribution defined by a HMM [21]. A HMM $\lambda$ is defined in equation 1.

[TABLE]

• •

$S=\{PLC_{1,1},...,PLC_{i,j}\}$,

• •

$V=id(S)$,

• •

$A\in\mathbb{R}^{(i\cdot j)\times(i\cdot j)}$, the state transition matrix,

• •

$B\in\mathbb{R}^{(i\cdot j)\times(i\cdot j)}$, the observation matrix,

• •

$\pi\in\mathbb{R}^{(i\cdot j)}$, the starting distribution describes the probability $\pi_{n}=P(X_{1}=s_{n})$ of the first state to be $s_{n}$

$i$ is the Number of IPCs and $j$ the Number of PLCs per IPC. $id()$ is a function that maps a PLC to a unique identifier as each PLC only outputs its identifier.

V Analysis and Results

The model introduced in section IV was implemented with specific parameters:

• •

$3$ products being produced by

• •

$5$ IPCs controlling

• •

$4$ PLCs each

The products are chosen randomly, following a Markov process [22]. The possible transitions are shown in figure 1, the corresponding transition probabilities are shown in table II. In total, 24 different setups were simulated and analysed. These setups consist of two ways the attacker reprograms the PLCs, three different numbers of malicious actions introduced, and four ways the IPCs are reconfigured and reprogram the PLCs due to the change of product. The simulation setups are an abstraction of real-world behaviour. As explained in section III, an operator knowing every transition of her system would be able to detect any attack, unless the attacker also knew the transitions and was able to mimic this behaviour. In this case, the operator could only tell valid from malicious attacks by consulting the necessity of changing states due to changes in production.

The goal of this simulation is to derive the importance of context information for detection of various malicious traffic in different kinds of valid traffic. We first created a set of sequences of valid actions and transitions. The transitions were saved in one, respectively three matrices. The first matrix contained all transitions, regardless of the product to be produced. The three matrices contained the transitions for the correlating product, introducing the notion of context. In the real world, this kind of information is easily accessible via Manufacturing Execution System (MES) and Enterprise Resource Planning (ERP), as, for example, proposed in [23]. First, the matrices were initialised by $10\,000$ cycles of product change. Normalisation was applied to the matrices. Then we inserted first one, then two and finally four malicious packets into each sequence of the set. After that, the transition probabilities of the malicious actions were calculated. The probabilities of the attacks, as well as the valid actions, were compared to a threshold varying from [math] to $1$ in steps of $0.05$. This allowed for the calculation of Receiver Operating Characteristics, as well as the f-measure. The f-measure, or F1-score, is a metric to determine the quality of a classifier [24], as written in equation 2.

[TABLE]

One of the most noteworthy properties of a Markov model is its inability to contain memory of states before the previous one, the so-called Markov property. This means that each transition probability only depends on the previous state, completely independent of its predecessors before the current state. This behaviour is noted mathematically in equation 3. The transition probabilities of each state transition only consider the previous state. where $p$ is a matrix of transition probabilities, $P$ the probability of an event, $X$ an event, $s$ a state and $t$ a time step.

[TABLE]

Each sequence of malicious programming actions was considered an attack and should have been found. It was, however, sufficient to detect one malicious segment within a sequence. In reality, finding only one of a series of malicious programming actions would be sufficient to raise awareness for an attack. First, we analyse attacks that are introduced to several scenarios in static order in subsection V-A. Then, attacks that are introduced into the same scenarios in different order are examined in subsection V-B. The results are presented in subsection V-C.

V-A Attacks in Static Order

In these setups, the attacks always were in a static order. In turn, one, two and four attacks were introduced into four different kinds of valid actions.

Setup 1

The IPCs that are reconfigured for each product are shown in table I. If a product is selected, the IPCs are reconfigured in their numerical order. The products are randomly chosen with probabilities as listed in table II, Upon reconfiguration, each IPC reprograms its corresponding PLCs. This reprogramming process is depicted in figure 2, the according transition probabilities are listed in table IV. The given probability distribution allows for some PLCs to not be reprogrammed. This behaviour is supposed to create a small uncertainty that, in reality, could arise from maintenance or reprogramming steps that were undergone. Still, one major difference between office- and industrial-IT lies in the repetitive traffic as described in section I. This repetitive behaviour is modelled by reconfiguring the same IPCs in the same order for each product and the order of reprograming of the PLCs.

The ROCs for one malicious action in each sequence can be found in figure 3. It can be seen that they are 1 almost instantly. This is due to the fact that the valid behaviour is relatively uniform, making it easy to detect malicious activities. For two and four malicious actions in each sequence, the ROC starts even higher.

In addition to the ROC, the f-measure is shown in figure 4. The difference of f-measure for context-less and context-based approach is depicted as well.

Because of the higher number of false positives for the context-less approach, the f-measure of the context-based approach is better in for certain thresholds. With increasing threshold, however, the difference decreases as an increasing threshold leads to less transitions that fit. The difference of f-measures looks similar for all numbers of malicious actions in this setup.

Setup 2

In this setup, the same IPCs as in the previous one are reconfigured when switching products. They are summarised in table II. They, however, reprogram their corresponding PLCs always in strict order. The ROC immediately reaches close to one, the diagram is therefore omitted. The f-measures are shown in figure 5. In the pictured scenario, four attacks were introduced into every sequence. The other scenarios look very much alike.

Sometimes, the f-measure of the approach without context seems to be greater than one. This is a result of plotting inaccuracy, as the f-measure always results to a value between zero and one.

The strict order of valid actions makes it very easy to distinguish malicious from valid actions, leading to such high true positive values of both approaches. Furthermore, the increase of the difference towards high thresholds results in the faster increase of false positive detections for the context-less approach. While both methods perform very good in terms of true positives, the context-based method is better with respect to choosing less false positives.

Setup 3

In this setup, each change of products led to reconfiguration of all IPCs and thus reprogramming of all PLCs. The PLCs in turn are reprogrammed in strict numeric order. However, the order of reconfiguration varied, depending on the product. It can be found in table V.

Again, the ROC starts with one due to the high true positive rate. There is, however, a significant deviation in the f-measure, as exemplarily shown in figure 6 for four attacks per sequence.

The difference in the f-measures indicates an outperformance with respect to false positives by the context-based approach.

Setup 4

In this setup, upon change of product, each IPC was reconfigured, as described in table V. The order of reprogramming the PLCs was completely random. In figure 7, the ROCs for all numbers of malicious actions, with and without the notion of context, respectively, are drawn. It can be seen that there is a break even point, where the curves align. Before that, detection with the notion of context outperforms detection without consideration of context.

The f-measures, depicted in figure 8 for one malicious action per valid sequence, indicate a lower performance than in the previous setups. The difference is located in the low thresholds, due to the better performance with respect to false positives of the context-based approach. The comparatively worse performance is a result of the randomness of valid actions.

V-B Attacks in Random Order

In these setups, the attacks always were in random order. In turn, one, two and four attacks were introduced into four different kinds of valid actions.

Setup 1

The reconfiguration pattern of IPCs is similar to the one described in table I of Setup 1 in subsection V-A, with product transition probabilities as described in table II. The reprogramming pattern of the PLCs is shown in figure 2. the according transition probabilities can be found in table IV. The results are similar as well, the ROCs saturate almost from the beginning, as can be seen in figure 9.

There is a small difference, however, in the f-measure of detection with and without the notion of context. As shown in figure 10, the detection without context slightly outperforms the detection without context in threshold ranges between $0.05$ and $0.3$.

Setup 2

As in Setup 2 of subsection V-A, upon reconfiguration of an IPC, according to tables I and II, all its PLCs are reprogrammed in strict order. The ROCs of one, two and four attacks per sequence start with one, depicting them is therefore of little interest. Due to the false positives, however, the f-measure values differ, especially with threshold greater than $0.2$. This behaviour is shown in figure 11.

Setup 3

As with Setup 3 in subsection V-A, in this setup, all IPCs are reconfigured when the product is switched. The order of reconfiguration for each product can be found in table V. The corresponding PLCs are reprogrammed in numeric order, respectively. The ROCs converge immediately. There can, however, be found significant differences in the f-measures, as shown in figure 12. Especially the area between $0.4$ and $0.6$ shows a difference due to false negatives, where the detection with notion of context outperforms the detection without notion of context.

Setup 4

In this Setup, the order of reconfiguration of IPCs again depends on the product as shown in table V. The order of reprogramming the PLCs, however, is random for each IPC respectively. The ROCs for one attack per sequence can be seen in figure 13.

Up to a threshold value of $0.2$, the detection with consideration of context performs significantly better. This can also be seen in the f-measures, as depicted in figure 14.

The ROC for detection without context seems to decrease in figure 13. This is mathematically impossible and is due to an inaccuracy in plotting the graph.

V-C Results

Throughout the experiments, detection with consideration of context always performs better than detection without consideration of context. This outperformance can in some cases be derived from the faster increasing ROC. In any case, the f-measure of context-based detection is higher and decreasing slower than the f-measure of non-context-based detection. Therefore, the false negative rate of context-based detection is lower than the false negative rate of context-less detection.

Furthermore, it shows that the quality of detection, in terms of ROCs, depends on the randomness of the valid actions. If only strict patterns occur, detection is easy. However, if there is a level of uncertainty, detection becomes more difficult, as the transition matrices become more ambiguous.

Another interesting result lies in the discovery that the order of attacks has very low influence on the detection rate. This is probably because of the fact that the transition from valid action to malicious action, and vice versa, serves as a distinguishing event. Transitions within the malicious actions are not necessary to detect them.

VI Conclusion

Industrial systems and networks are typically more uniform and deterministic than classic internet networks. This fact can be used to aid in intrusion detection. If context information is considered in conjunction with network information, intelligence about the presence or absence of malicious activities can be gained more easily. Even though the proposed experiment was a simplification of a real industrial network, the effect is noteworthy. Especially false positives are significantly lower with than without notion of context. In real-world applications, this is a highly important factor, as each false positive will lead to investigation and consume time and effort, resulting in high costs. A low false positive rate is therefore critical for acceptance of an detection algorithm.

For our future work, we plan on creating a more sophisticated simulator on a packet base, so that frequency, content, length and other properties of network traffic can be observed. In addition to that, we plan on integrating host-based information, such as log-entries or settings of production tools, into the detection solution. This work suggests that this will lead to an increase in detection with a decrease in false positives.

Acknowledgment

This work has been supported by the Federal Ministry of Education and Research of the Federal Republic of Germany (Foerderkennzeichen KIS4ITS0001, 16KIS0311, IUNO). The authors alone are responsible for the content of the paper.