The Dos and Don'ts of Industrial Network Simulation: A Field Report

TL;DR

This paper reviews industrial network simulation tools by analyzing use cases and requirements, providing guidance on selecting appropriate simulation environments to improve efficiency in industrial network design.

Contribution

It identifies industrial use cases, derives requirements, and maps existing simulation tools to these needs, offering practical insights for industrial network simulation.

Findings

Industrial networks are complex and require tailored simulation tools.

Existing simulation tools vary in capabilities and suitability.

Mapping tools to use cases improves selection and application.

Abstract

Advances in industrial control lead to increasing incorporation of intercommunication technologies and embedded devices into the production environment. In addition to that, the rising complexity of automation tasks creates demand for extensive solutions. Standardised protocols and commercial off the shelf devices aid in providing these solutions. Still, setting up industrial communication networks is a tedious and high effort task. This justifies the need for simulation environments in the industrial context, as they provide cost-, resource- and time-efficient evaluation of solution approaches. In this work, industrial use cases are identified and the according requirements are derived. Furthermore, available simulation and emulation tools are analysed. They are mapped onto the requirements of industrial applications, so that an expressive assignment of solutions to application domains is given.

Tables

Table 1. Table 1. Requirements Mapped to Solutions

Solution: Requirement:	S1	S2	S3	S4
Timing Precision	$✓$	$✓$	$✓$	$\bullet$
Flexibility	$\times$	$✓$	$✓$	$\bullet$
Real World Interaction	$✓$	$✓$	$✓$	$✓$
Trusty Physical Interaction	$✓$	$\bullet$	$\bullet$	$\times$
Adaptability	$✓$	$✓$	$\bullet$	$\bullet$
Ease of Use	$\times$	$\bullet$	$✓$	$\bullet$
Simulation/Emulation	Both	Both	Sim	Emu
Levels of Automation	1-5	1-5	2-3	3-5

Full text

This is a preprint of a work published in the Proceedings of the 2nd International Symposium on Computer Science and Intelligent Control (ISCSIC 2018). Please cite as follows:

S. D. Duque Anton, D. Fraunholz, D. Krummacker, C. Fischer, M. Karrenbauer, and H. D. Schotten, : “The Dos and Don’ts of Industrial Network Simulation: A Field Report.” In: Proceedings of the 2nd International Symposium on Computer Science and Intelligent Control (ISCSIC 2018), 2018, pp. 6-13.

Simulation, Industrial Internet of Things, Survey, Industry 4.0, Network

††journalyear: 2018††copyright: acmlicensed††conference: The 2nd International Symposium on Computer Science and Intelligent Control; September 21–23, 2018; Stockholm, Sweden††booktitle: The 2nd International Symposium on Computer Science and Intelligent Control (ISCSIC ’18), September 21–23, 2018, Stockholm, Sweden††price: 15.00††doi: 10.1145/3284557.3284716††isbn: 978-1-4503-6628-1/18/09††ccs: Computer systems organization Embedded systems††ccs: Computer systems organization Redundancy††ccs: Computer systems organization Robotics††ccs: Networks Network reliability

1. Introduction

Industrial networks have been proprietary systems, set up for each specific application scenario. With the rising importance of the Industrial Internet of Things (IIoT), standardised and extensive interconnectivity gains importance. The influence of networking in production is increasing, control and maintenance tasks can be performed remotely (karrenbauer2018, ). Well-established communication protocols, such as EtherCAT, Modbus, Modbus/TCP, PROFINET and OPC-Unified Architecture (OPC-UA), as well as Commercial Off The Shelve (COTS) hardware reduce the configuration and customisation effort in setting up networks. On the other hand, the increasing integration of communication into automation creates a deeper penetration of industrial applications by communication and computational devices. Recently, wireless protocols, such as WirelessHART, ZigBee and LoRa add to the variability of solutions. The increasing importance of networking for industrial applications justifies a rising need for network simulation tools of industrial environment. It is necessary for planning of network structures, for the evaluation of timing criteria, for adaption of control algorithms in Software-Defined Networking (SDN), and also for security applications. This work aims at providing an overview of existing methods for simulation and emulation, enabling users to chose the best suited method and to create the best fitted environment.

The remainder of this work is structured as follows. In section 2, an overview of works analysing and categorising simulation environments is provided. After that, relevant uses cases for simulation of industrial networks are introduced in section 3. Requirements derived from these use cases are explained in section 4, solution approaches are presented in section 5 and a mapping of these solutions to the requirements is provided in section 6. The work is concluded in section 7.

2. Related Work

Due to the novelty of this topic, not much work has been done to evaluate simulation tools for industrial applications. Terzi and Cavalieri considered simulation as a part of supply chain management and a tool for change management (Terzi.2004, ). Apart from that, work has been done in evaluating well-established simulation tools for their performance. Weingartner et al. compare the performance of five network simulators (Weingartner.2009, ), while Flores Lucio et al. analyse accuracy of ns-2 and OPNET (Flores_lucio.2003, ). Pan (Pan.2008, ), as well as Siraj et al. (Siraj.2012, ) consider simulator environments and testbeds as means to save money, time and effort in network planning. They present several tools and compare advantages and disadvantages. The work of Göktürk addresses the same topics, after systematically distinguishing between testbeds and emulation (Gokturk.2007, ). Furthermore, there are surveys addressing specialised fields of network simulation. Kösal analyses four well-known simulators that support wireless networks (Koksal.2008, ), Naicken et al. consider peer-to-peer network simulators (Naicken.2006, ), Martinez et al. address vehicular ad hoc networks (Martinez.2011, ), and the work of Sarkar et al. is about simulation of telecommunication networks (Sarkar.2011, ). In addition to these use cases, several works address simulation tools for Wireless Sensor Networks (Sundani.2011, ; Imran.2010, ; Korkalainen.2009, ; Christin.2010, ; Yu.2011, ). WSNs are becoming more relevant with the increase in embedded computing power. Distributing WSNs contains a relatively large organisational overhead, justifying the use of sound simulation environments.

3. Use Cases for Industrial Network Simulation

In this section, the different use case scenarios motivating the appliaction of simulation environments are described in subsection 3.1. Developments in technologies of interconnectivity, as well as the integration of Internet of Things (IoT) devices into the industrial environment create novel types of industrial networks, e.g. the IIoT. After that, the simulation scopes are described in subsection 3.2. Some use cases require a high-level view of a network without the need for simulating a detailed behaviour, while others need well-defined traffic packets. Finally, the difference between simulation and emulation is explained in subsection 3.3.

3.1. Use Case Scenarios

In this subsection, the prototypical scenarios of industrial use cases that could benefit from simulation are described.

3.1.1. Network Management Algorithms

A new set of use cases is introduced due to Industry 4.0 (3gpp2017, ), entailing new challenges to industrial communication networks. The derivation of the challenges are introduced in (DenKrChFiAR2018, ). These challenges create demand for a sophisticated network management that, on one hand, is capable of handling a heterogenous landscape of industrial communication protocols. On the other hand, it needs to satisfy requirements like ultra low latency, determinism, cross domain communication, dependability, flexibility and security. There are innovative technologies at hand that meet these requirements, sometimes in conjunction with one another. Examples for these technologies are SDN, which provides flexibility and scalability to networks, and Time-Sensitive Networking (TSN), which allows for determinism of the network, as well as Network Function Virtualisation (NFV), which can control dependability relevant parameters in a centralized management system. The modular combination of technologies can cause interdependency, inferring the danger of a live-lock behaviour in a control or management system. Due to this issue, sophisticated algorithms capable of combine technologies and resolve interdependency are required. Network management is needed in several points of the work flow. First of all, integration of TSN and SDN into a network can not be performed on a single switch. Complex network scenarios have to be set up to be developed and evaluated iteratively. Algorithms for network management require these underlying technologies in order to be developed for a network. The above mentioned circumstances yield to sophisticated requirements on the test environment in terms of flexibility, openess of interfaces and scalability. Due to this a simulation is a preferable way to deploy the test environment.

3.1.2. Digital Factory

The Digital Factory is a use case described by Stef at al. (STEF2013451, ). It focuses on the simulation of a production unit in an industrial environment. In doing so, the debugging can be executed in advance of the setup and the deployment of the production unit can be accelerated. Besides the simulation of the production unit, a simulation of the network is crucial to provide an appropriate dimensioned network capable of meeting all technological demands. Furthermore, it is possible to deploy major changes in network settings in a simulated environment and run through the debugging process, before implementing it on the running hardware. In this way, possible network crashes can be avoided. In this case the advantages of a simulation concerning deployment speed, cost reduction are the main arguments for their usage. One characteristic of the digital factory is a flattened automation pyramid. Control is not necessarily introduced from a higher layer anymore. Instead, entities on the same layer may communicate through network boundaries. Many of such use cases include the necessity for real-time communication, that consequently has to be addressed by the simulation tool chosen for the scenario.

3.1.3. Honeypots

Deception is a well established method in many domains. In IT security it is known as Deception Technology. The most noteable implementation are Honeypots. HPs, in their very essence, mimic resources such as Programmable Logic Controller (PLC)s or Supervisory Control and Data Acquisition (SCADA) systems. The deceptive capabilites rely on the quality of simulation, more precise and comprehensive imitations allow for a deeper insight into the interaction and therefore the motivation, expertise and arsenal of hostile entities. As these insights are valueable for prevention (Fraunholz.2018b, ), detection (Fraunholz.2018, ) and attribution (Fraunholz.2017e, ; Fraunholz.2017d, ; Fraunholz.2017f, ), the use of HPs is promising for industrial security. They are also able to cope with the most pressing challenges, such as durability, latency, heterogeneity and encryption. The simplest industrial HPs support common industrial protocols, such as Bacnet, OPC-UA or modbus (Rist.2015, ). For those protocols, that are frequently employed in enterprise and industrial environments, a large selection of HPs is available. Examples are HTTP/HTTPS (Provos.2004, ), FTP (CarnivoreProject.2009, ) and Telnet/SSH (Oosterhof.2014, ). Since productive systems are usually interconnected, so are HPs. In this case, they are referred to as Honeynets. The infamous honeyd (Provos.2004, ) is an example for a HN implementation. Besides communication capabilites, system simulation is also an integral part of deception. Recent implementations use IMUNES/Conpot (Kuman.2017, ), Matlab/Simulink (Litchfield.2017, ) or gridlabd (GridPot:SymbolicCyberPhysicalHoneynetFramework.2015, ; Redwood.2015, ) for system simulation.

3.1.4. Industrial IT Security

The increasing interconnectivity and rising connection of industrial to public networks leads to an enlarged attack surface of these networks (Duque_Anton.2017a, ). In contrast to home and office IT, industrial networks have been set up with little thought of security. Since this resulted in, sometimes drastic, attacks in recent years, this paradigm is changing. Security solutions, such as industrial firewalls and Security Information and Event Management (SIEM) systems are being developed. Furthermore, there is a large research interest in detecting anomalies and intrusions in industrial network traffic. Context information has proven to be a valuable indicator (Duque_Anton.2017c, ), but is not trivial to obtain (Duque_Anton.2017b, ) Apart from IT security reasons, anomalies can be an indicator of misconfiguration or malfunction, motivating the need to detect them in a timely manner. There are some works regarding anomaly and intrusion detection in industrial network data (Duque_Anton.2018, ). Several surveys cover this topic, such as the work of Garcia-Teodoro et al. (Garcia-Teodoro.2009, ) and Bhuyan et al. (Bhuyan.2014, ). Yang et al. give a brief introduction of these techniques for the domain of SCADA systems (Yang.2006, ). Meshram and Haas published a roadmap of machine learning based anomaly detection in industrial networks, containing a simulation environment, as well as a semantic description of content (Meshram.2016, ). Kleinman and Wool present a model of the Siemens S7 protocol for intrusion detection and forensics (Kleinmann.2014, ). Critical infrastructures and industrial environments are considered in the work of Hadziosmanovic et al. (Hadziosmanovic.2011, ). A framework that detects malicious and undesired actions is presented. Deriving features that can be used to distinguish valid from malicious traffic is the first step in applying an intrusion detection algorithm. Mantere et al. look into the derivation of features from Internet Protocol (IP) traffic in an industrial environment (Mantere.2013, ). Deterministic properties of industrial control systems, as well as the usability of this feature for anomaly detection in an industrial environment, is researched by Hadeli et al. (Hadeli.2009, ). Most of these algorithms need a large amount of data in order to be properly trained. In contrast to home and office network traffic, industrial network traffic is still rare. This is an issue Morris and Gao describe in their work (Morris.2014, ). Therefore, they introduce an intrusion detection dataset that, however, consists of aggregated data instead of network packets. This renders is unfit for anomaly detection algorithms that are packet- or timing-based.

In general, due to concerns regarding privacy, confidentiality and effort, simulation will play a crucial role in intrusion detection for industrial systems. Real world data sets are difficult to obtain, mostly due to organisational issues. This means that research has to use synthetic data or data that has been created in research environment. The realism and consistency of this data is crucial, as the results of an industrial Intrusion Detection System (IDS) depend on them. Flawed data could lead to good results on the test data, but weak performance in a productive environment. As the demand and necessity of industrial intrusion detection will increase over the next years, the need for data will rise.

3.2. Scope of Simulation

Apart from the use case scenarios as described above, the scope of the simulation is of interest. First, the scope in terms of automation level is discussed. After that, the scope in terms of geographical spreading is examined.

3.2.1. Level of Automation

In the 1990’s, so-called automation pyramid has been introduced. It is depicted schematically in figure 1.

The different levels of a hierarchical industrial network are explained with this automation pyramid, as there are different requirements for different levels. On layers 1 and 2, for example, real time communication is often required. In addition to that, a physical model of the production system is required for layer 1, which is usually difficult to implement. Layers 4 and 5, Enterprise Resource Planning (ERP) and Manufacturing Execution System (MES); systems, that control the setup and production processes of automation devices, on the other hand, are mostly implemented using standard TCP/IP networks. The SCADA system on layer 3 commonly employs non-real time Ethernet-based protocols.

3.2.2. Geographical Scope

Per Se, networks are employed to connect different systems and enable communication. There are several characteristics of networks. A typical set of scenarios is depicted in figure 2.

Depending on the scenario, simulation of traffic in different parts of the shown networks can be relevant. A single connection between the PLCs in the production part of the figure can be of interest, e.g. when the characteristic of a certain antenna is to be determined. For network management, a whole site, including connection between office and production area can be under investigation. Furthermore, the connectivity possibilities over public networks can be analysed, e.g. for security analysis.

3.3. Simulation vs. Emulation

In order to categorise solutions, a terminology has to be introduced first. Generally, there is a semantic difference between simulation and emulation, even though these terms are often used interchangeably. In this work, we will generally use the term “simulation” for the sake of readability, since many tools can be used for both.

Generally, simulation aims at mimicking the states of a system (Cellier.1991, ), while emulation aims at reproducing the behaviour (KB., ). This difference often becomes relevant in the type of experimental setup: As discussed in the following subsection, simulation is usually done on one computer. That means that any physical characteristic has to be modeled. The goal is to create a model of the components under investigation and imitate the behaviour, as well as internal states as accurately as possible. Sometimes, this leads to a high effort in creating the simulation model. Emulation, on the other hand, can be done on real network infrastructure with devices imitating other devices. If only the transmission behaviour is of interest, then the nature of devices creating the traffic is irrelevant. Emulation therefore has the advantage that its output is, by design, as accurate as the productive system, since the physical environment is used. On the downside, it usually has a higher requirement in devices and physical space.

4. Requirements for Industrial Network Simulation

In this section, each of the use case scenarios, as described in subsection 3.1, is analysed. The requirements on a simulation or emulation environment are derived from the characteristics and explained. Furthermore, the scope in terms of abstractivity and geographical extension, as described in subsection 3.2, is considered.

4.1. Network Management Algorithms & Digital Factory

Industrial production is gaining flexibility. Future industrial production systems are getting an increasingly dense interconnectivity that has to serve a rising amount of network traffic, inflicting stringent requirements on the network. This raises the necessity of an autonomous operating and automatic self-organising-network to be able to master the complexity. A possible solution for a Network Management Concept is presented in (DenKrChFiAR2018, ). A Simulation has to mimic the real-world behaviour as accurately as possible, when it comes to metering especially performance dependent aspects of such systems. Even more important is an accurate replica of industrial network systems for development of new solutions. In case of analysing the fundamental behaviour of transmissions on a network instead of functional high-level solutions, accuracy of the simulation is crucial. The scope of this simulation sometimes extends to the characteristics of singular packets on the physical transmission medium, e.g. in safety packets that inflict hard real-time requirements.

4.1.1. Timing Precision

A key demand for digital factories is the ability to grant certain transmission guarantees, e.g. maximum transmission time or reliability. While developing new transmission mechanisms, protocols or overlaying methods influencing transmission procedures, it is important to have the ability to measure the timing behaviour of transmissions under the given circumstances. The essential point is that not only the functional aspect of macroscopic or cumulated operations has to be evaluated, but rather the fine-grained runtime performance of a set of operations. This requires the simulation of a highly precise and representative time base on which a realistc imitation of the transmission time behaviour can be achieved. For protocols and low level methods, it has to be evaluated if they grant the ability to meet requirements, for instance transmission deadlines in general. Higher level methods need to prove that they properly interact with underlying or cooperating procedures without disrupting.

4.1.2. Flexibility / Extensibility

A huge variety of differing communication technologies, protocols and vendor-specific proprietary solutions is assembled in industrial networks. In order to address this characteristic by a simulation tool, it must provide the possibility to integrate novel modules above the physical level, such as protocols or medium access methods. Additionally, when working on new communication technologies or fundamentals of them, e.g. new waveforms of wireless transmissions, the physical and transmission models of the simulation have to be adapted. For example, the introduction of differing channel models to adjust the simulation to different environments can be of value.

4.1.3. Real-World Interaction

Often, existing parts of a network infrastructure should be integrated into a simulation. This can be due to the extension of a network, or the effort it would take to create a model of devices or infrastructure already existing. Basically, network simulation is predestined to interact with external, real-world domains. In order to efficiently do so, simulation environments need to provide interfaces to communicate with protocols and devices they are modelling.

4.1.4. Trusty Physical Imitation

The ability to accurately measure physical influences on networks, to enable a realistic comparison of different communication technologies, is of high value. These physical influences need to be modelled with sufficient accuracy for the simulation to provide results that will not deviate from any real-world application. Usually, not all parameters are of equal importance. An application example is the comparison of fading effects in narrow-band and ultra-wide-band wireless communication.

4.2. Honeypots

In contrast to other use cases, HPs only rely on the perception of the entity that it intends to deceive (Fraunholz.2017g, ). It is therefore important to define the attack vector and simulate the corresponding observations a hostile entity may encounter. For example, if the attack scenario is most likely network sniffing, the network packets should be generated in accordance to the communication stack of the system that is simulated. From a technical perspective, there is a trade-off between quality of simulation and handling of the systems. Systems with a more realistic imitation tend to be more complex to monitor and secure as there is a wider variety of possible actions. There is an abundance of examples, where systems that provide full-fledged operating systems are suffering from easy detection, relay attacks and monitoring circumvention (Fu.2006, ; Chen.2008, ; Mukkamala.2007, ; Dornseif.2004, ; Holz.2005, ). To mitigate detection attempts, it is of significant importance to enable a dynamic configuration, deployment and management of such systems (Fraunholz.2017b, ; Fraunholz.2017, ). However, not only technical requirements need to be considered when applying such technologies. It must be ensured that the application is conform with law of the corresponding jurisdiction (Fraunholz.2017i, ).

4.3. Industrial IT Security

Industrial IT security is a very broad area with an abundance of requirements. In this work, an effort is undertaken to generalise them while being accurate enough to derive possible solutions.

4.3.1. Injectability of Malware

The first and foremost requirement of industrial network data generation for IT security is the possibility to inject malicious traffic. This malicious traffic has to fit the environment and correctly represent the instantiation of a possible attack vector. In emulation environments with real network infrastructure, executing malware with common penetration testing tools, such as metasploit, is an option. This, however, is not possible in network simulations running on one machine. If simulation is chosen, the malicious behaviour needs to be modeled and injected in a valid and sound fashion.

4.3.2. Variability

Due to the inherent unique nature of most industrial networks, a versatile and adaptive data generation environment is requried. In simulation, as well as emulation, adapting or creating new scenarios requires reconfiguration and possibly creation of new models. Tools need to be able to adapt to these changes.

4.3.3. Viability and Soundness

Especially when employing methods of machine learning, many algorithms are prone to overfitting. That means they focus on parameters that have little to nothing to do with the actual traffic, but are artefacts of data generation. The DARPA KDD cup ’99, for example, contains one feature that is specific to all malicious instances, but solely stems from the simulation model. If IDSs are tested on such data, they lack performance once they are integrated into real world processes. To prevent focussing on invalid features and overfitting, data generation has to be sufficiently sound and compatible to real world data. This is commonly considered as the most difficult task in data generation, especially since a user often cannot determine which feature has which influence on an algorithm.

4.4. Requirements Summary

So far in this section, the requirements on all of presented use cases have been described. In order to map the possible solutions, as described in section 5, the requirements are summarised. In general, there are five requirements for industrial network simulation that are listed below:

• •

Timing precision describes the accuracy and timeliness of traffic, which needs to be highly precise for TSN.

• •

Flexibility in setting up and changing simulation characteristics

• •

Real world interaction, the capability to be integrated into existing productive environments

• •

Trusty (physical) imitation describes the soundness and correctness of simulation

• •

Adaptability describes the potential to be adjusted and mapped to novel scenarios

5. Solution approaches

In this section, several solutions are proposed and their advantages and disadvantages discussed. The solutions presented in this section are: Application specific solutions, ns-3, OMNeT++, and Mininet.

5.1. S1: Application Specific Solutions

Creating an application specific simulation tool allows for the highest level of flexibility, on the cost of the most effort needed. They can be either simulation or emulation environments, or a mix. Furthermore, real components can be used to model the system. Real network parts can be used with simulated devices, for example. For industrial IT security, Lemay and Fernandez provided a data set from an application specific simulation (Lemay.2016, ). Wang et al. provide a simulation environment for SCADA security assessment (Wang.2010, ), as does Seidl (Seidl.2015, ). and Siaterlis et al. (Siaterlis.2013, ), as well as Genge et al. (Genge.2012, ) introduce testbeds for research in industrial IT security.

5.2. S2: ns-3

Ns-3 is the name of an event-based open source network simulator which can be used for researching on IP and non-IP based networks. It aims at providing simulation models that are as realistic as possible and can be used as a network emulator, i.e. interconnected with real world devices. Ns-3, being an event-based simulator, already provides a time base and many other modules, including a WiFi PHY and MAC layer implementation, wireless channel models and protocol stacks. Therefore, ns-3 is a sound choice for simulating the inter-connectivity of different networks, especially but not limited to wireless networks, which makes it appropriate for modeling the use case scenarios ”Network Management Algorithms“ and ”Digital Factory“. Ns-3 provides different simulation models. In a closed simulation, ns-3 traverses through the simulation whilst it maintains a changing event-list. Throughout runtime, ns-3 processes one event after the other until the event-list is empty or a specified point in time for the simulation end is reached. Events are caused explicitly by instructions from the simulation program or can be raised by other events during runtime (possibly infinitely). Alternatively ns-3 supports a real-time scheduler that facilitates interaction with real systems.

Ns-3 itself is written in C++ and for its use provides libraries for the programming language C++. Accordingly, ns-3 simulations are in fact programs, written in C++, using the ns-3 libraries. An ns-3 simulation defines the involved network devices and per device the used network stack and eventually running applications. Between the devices the communication technology is specified with an explicitly chosen channel model. If desired, one could implement custom channel models, completely new underlying calculation methods or new physical influences and thereby introduce new technologies. Admittedly, doing so would require a high effort. The out-of-the-box implemented transmission channel replications of ns-3 move somewhere between realistic accuracy and low computational expense but are adequately representative, like evaluated by (boeingNS3ChanEval, ).

Likewise ns-3 can be extended with new protocols or any other module that incorporates somewhere in the processing of data through a network stack, across a network or if desired completely local on one machine. Network devices in ns-3 aren’t full-featured devices inclusive operating system (like it would be in the case of mininet). Rather every cooperating piece has to be specified. As its purpose to be a network simulator, the most common networking features are already available in ns-3 to be deployed to simulated devices. It is a substantial property of ns-3 that it is open source and high-grade modular, which creates a maximum on flexibility with the choice of willingness for effort up to the developer of a simulation. Many features and accuracy are already available in ns-3. If one desired is not, it is likely that the implementation of it would be possible with a smooth incorporation to distinct ns-3 simulation modules. Ns-3 indeed creates network packets and passes them like real devices would. As a consequence, these packets can be treated like real ones, for instance can be sniffed using libpcap, analysed, logged, malformed or modified. Ns-3 offers the ability to capture and process packets live, i.e. during the running simulation or to log them for later use in various formats, like .pcap or plain ASCII.

5.3. S3: OMNeT++

The Objective Modular Network Testbed in C++ (OMNeT++) (OMNeT++.2015, ) provides a C++-based network simulation platform, even though “OMNeT++ is not a network simulator itself”. Devices and networks are described within the environment provided. They are then connected and communicate according to a given pattern on the physical layer, based on a discrete event simulator. Integration of higher layers is possible by employing so-called frameworks, that implement protocols and functionality and can be used by the OMNeT++ environment. Due to this extendibility, any functionality can be integrated into the OMNeT++ environment. Wireless, as well as real time applications can be simulated. Furthermore, the use of OMNeT++ as an emulation testbed over a real network is possible by employing sockets. In general, the functionality is similar to ns-3. There are graphical user interfaces available for OMNeT++, making introduction easier. Furthermore, the licensing model is different: While there is an open source license for academia, industrial users have to purchase a proprietary OMNeT++ distribution.

5.4. S4: Mininet

Mininet (mininetWeb, ) is a network emulator available for Linux Operating Systems. It is more high-level and oriented on using a network (e.g. for development or testing of applications or control algorithms) instead of researching on low-level network topics itself (such as channel access schemes or synchronisation protocols for instance). On boot, it creates a live interactive emulation environment, using Linux Kernel utilities. Mininet creates a network of virtual hosts, switches, controllers and links, within whose the hosts run standard Linux network software. Mininet uses process-based virtualization and network namespaces to run many111the developers successfully booted up to 4096 hosts and switches on a single OS kernel. The network connections are realized through virtual ethernet (veth) pairs and pipes. Mininet out-of-the-box supports kernel and user-space OpenFlow switches and also controllers to control the switches. Mininet is very light-weight compared to many other virtualisation workflows. As shown in (a19-lantz, ), it can yield an efficient use of time and resources, providing interactive prototyping, scalability and straightforward sharing and collaboration for close-to-real-world developing and testing. For controlling the emulation setup, topology and conducted actions Mininet offers on one side an extensible Python API through which predefined actions, like network creation can be done. On the other hand there is also a command-line interface that allows controlling the environment as well as every single device live during runtime.

Due to the fact every Mininet host can be assumed as being a full-featured Linux-device with ordinary access, the actual development topic does not necessarily have to be integrated in the emulation environment but rather can be independently worked on and just be called from within the emulated network, exactly how using a real network. Furthermore this applies also for concurrent access of several developers.

Since Mininet runs on Linux, uses Linux utilities, acts like Linux and does not aim to mimic any physical effects, it does not span any unique calculation basis and thus doesn’t create an own isolated virtual time-base (”in it’s own isolated world”) that is depending on the computational model or something like an internal Event-Handling. Rather it acts like a real network would, only the scalability of an emulated network depending on the computation power of the executing device. Concomitant is Mininet capable of interaction with real world devices. Using Network Interface Cards of the emulation executing device packets can be exchanged between virtual Mininet devices and the real world.

Whereas Mininets goal is to provide a network to work on, i.e. to emulate the behaviour of an interconnection and not to simulate the behaviour of a physical transmission, it does not work on any sophisticated channel or loss model. Mininet does indeed supply some parameters for the transmission connection to deliminate them from a perfect channel. For example one can limit the bandwidth or specify a statistical packet loss rate ($x\%$ packets lost) but that is not near as realistic as the opportunities made possible by low level network simulations. This fact essentially limits the representativity and accuracy of timing measurements for transmissions, when intended to work on lower layer solutions.

From its mentioned intention, Mininet is highly usable for functional development like application level communication or timing-uncritical SDN-algorithms but cannot suit performance related research that approaches how Data actually is transferred.

6. Solution Mapping

In this section, all solutions presented in section 5 are mapped onto the requirements as derived in section 4, with notes on simulation and emulation capabilites, as well as the scope that can be met. An additional requirement, ease of use, has been added, since commonly, usability is an important feature in industrial applications. This mapping is shown in table 1.

Solution S1, application specific solutions, provide the highest possible degrees of freedom for timing precision, real world interaction and adaptability. It is, however, neither flexible, as hardcoded solutions are usually hardly adjustable, nor easy to use, respectively implement. As it strongly depends on the singular case, it can be both simulation as well as emulation, and it can be placed on any layer of automation.

Solution S2, which is ns3, supports the use of a real-time scheduler which leads to a high achievable timing precision. This ability in conjunction with many models which are realistic enough to be used together with real-world network devices enables ns3 to be used as a real-time network emulator. Due to its open-source nature and the availability of many adaptable modules, the flexibility and adaptability of ns3 is high, although working in the ns3 core modules is effortful. The flexibility of ns3 arises also from its abstraction layer (e.g. node, net device, application, etc.) which allows the adaptation of the simulation on different layers. The physical interaction, in particular when it comes to wireless channels, is simulated using different models. The suitability and accuracy for different environments and use-cases follows from the accuracy of the underlying ns3 model, which is sufficient for a large number but not all of the imaginable situations.

Solution S3, OMNeT++, allows for good timing precision, is flexible and easy to adjust, as well as easy to use. The physical interaction on the other hand depends on the simulation model, with all the drawbacks of artifacts and inconsistencies. Furthermore, it is only easily adaptable to scenarios and models already implemented in libraries, novel methods take more implementation effort. It is a simulation environment, however, it could be used as an emulation tool. Physical sensor simulation is not possible, as well as ERP and MES are out of scope.

Solution S4, Mininet, does not possess an own time-base and its latency is limited by the resources provided by the host computer; deterministic timing is therefore not possible. It behaves like a Linux environment and its flexibility and adaptability is thus limited to the Linux world. Being an emulator, the interaction with real-world network devices is possible and commonplace. The consideration of physical aspects is limited to a few very generic possibilities to model the influence of a channel.

Additionally, it should be remarked that cost and licensing has not been considered explicitly, as it is not a functional requirement. Emulation in general is more costly than simulation, due to the hardware needed. Furthermore, while ns-3 and Mininet are open source, OMNeT++ demands for a license in case of industrial application. On the other hand, open source software usually contains so-called “copyleft”, forcing users to make tools using the software available.

7. Conclusion

In this work, four different kinds of scenarios in industrial applications are described. The requirements posed on each of those scenarios are derived and summarised. After that, commonly used solutions for network simulation are presented. The features they provide are mapped against the requirements, showing that most requirements can be met by one or another solution approach, even though there is no general solution meeting all requirements. The employment of a specific solution approach strongly depends on the use case and the kind of data needed. In addition to that, wireless networks require different simulation environments than wired networks, and locally small networks have a different scope than widespread networks with heterogeneous entities. While real world hard- and software provide the most realistic results, they are most difficult to set up, often negating the possible advantage with the effort needed.

Acknowledgment

This work has been supported by the Federal Ministry of Education and Research (BMBF) of the Federal Republic of Germany within the projects IUNO (KIS4ITS0001) & FIND (16KIS0571). The authors alone are responsible for the content of the paper.