Exploring how Component Factors and their Uncertainty Affect Judgements of Risk in Cyber-Security

TL;DR

This paper investigates how expert assessments of cyber-physical system components, including their inherent uncertainties, influence vulnerability evaluations, proposing a novel interval-based method to better quantify and utilize expert uncertainty in risk analysis.

Contribution

It introduces an interval-valued response format for capturing expert uncertainty and demonstrates its effectiveness in improving vulnerability assessment accuracy.

Findings

Uncertainty in expert assessments predicts overall component vulnerability.

Capturing uncertainty enhances the understanding of system risk.

Method is applicable to complex, real-world cyber-physical systems.

Abstract

Subjective judgements from experts provide essential information when assessing and modelling threats in respect to cyber-physical systems. For example, the vulnerability of individual system components can be described using multiple factors, such as complexity, technological maturity, and the availability of tools to aid an attack. Such information is useful for determining attack risk, but much of it is challenging to acquire automatically and instead must be collected through expert assessments. However, most experts inherently carry some degree of uncertainty in their assessments. For example, it is impossible to be certain precisely how many tools are available to aid an attack. Traditional methods of capturing subjective judgements through choices such as \emph{high}, \emph{medium} or \emph{low} do not enable experts to quantify their uncertainty. However, it is important to measure the range of uncertainty surrounding responses in order to appropriately inform system vulnerability analysis. We use a recently introduced interval-valued response-format to capture uncertainty in experts' judgements and employ inferential statistical approaches to analyse the data. We identify key attributes that contribute to hop vulnerability in cyber-systems and demonstrate the value of capturing the uncertainty around these attributes. We find that this uncertainty is not only predictive of uncertainty in the overall vulnerability of a given system component, but also significantly informs ratings of overall component vulnerability itself. We propose that these methods and associated insights can be employed in real world situations, including vulnerability assessments of cyber-physical systems, which are becoming increasingly complex and integrated into society, making them particularly susceptible to uncertainty in assessment.