Design Considerations for Building Credible Security Testbeds: A Systematic Study of Industrial Control System Use Cases

TL;DR

This paper systematically analyzes design factors for credible ICS security testbeds, emphasizing how attributes like expertise, objectives, and architecture influence trustworthiness and effectiveness in cybersecurity research.

Contribution

It introduces a comprehensive mapping framework for ICS testbed design considerations, enhancing their credibility and utility for security evaluations.

Findings

Design attributes impact testbed credibility

Expertise and objectives influence trustworthiness

Implementation and evaluation methods are critical

Abstract

This paper presents a mapping framework for design factors and implementation process for building credible Industrial Control Systems (ICS) security testbeds. The resilience of ICSs has become a critical concern to operators and governments following widely publicised cyber security events. The inability to apply conventional Information Technology security practice to ICSs further compounds challenges in adequately securing critical systems. To overcome these challenges, and do so without impacting live environments, testbeds for the exploration, development and evaluation of security controls are widely used. However, how a testbed is designed and its attributes, can directly impact not only its viability but also its credibility as a whole. Through a combined systematic and thematic analysis and mapping of ICS security testbed design attributes, this paper suggests that the expertise of human experimenters, design objectives, the implementation approach, architectural coverage, core characteristics, and evaluation methods; are considerations that can help establish or enhance confidence, trustworthiness and acceptance; thus, credibility of ICS security testbeds.