Empirical assessment of the perception of graphical threat model acceptability

TL;DR

This study empirically compares the acceptability of three graphical threat models for non-technical users, finding ADTs and CORAS broadly suitable, while highlighting the need for dedicated tools for Attack Graphs.

Contribution

It provides an empirical assessment of graphical threat models' acceptability among non-technical users, highlighting ADTs and CORAS as effective options.

Findings

ADTs and CORAS are broadly acceptable for non-technical users

Lack of dedicated tools may reduce Attack Graphs' perceived usefulness

Further research needed on Attack Graphs' acceptability and tool support

Abstract

Threat modeling (TM) is an important aspect of risk analysis and secure software engineering. Graphical threat models are a recommended tool to analyze and communicate threat information. However, the comparison of different graphical threat models, and the acceptability of these threat models for an audience with a limited technical background, is not well understood, despite these users making up a sizable portion of the cybersecurity industry. We seek to compare the acceptability of three general, graphical threat models, Attack-Defense Trees (ADTs), Attack Graphs (AGs), and CORAS, for users with a limited technical background. We conducted a laboratory study with 38 bachelor students who completed tasks with the three threat models across three different scenarios assigned using a Latin square design. Threat model submissions were qualitatively analyzed, and participants filled out a perception questionnaire based on the Method Evaluation Model (MEM). We find that both ADTs and CORAS are broadly acceptable for a wide range of scenarios, and both could be applied successfully by users with a limited technical background; further, we also find that the lack of a specific tool for AGs may have impacted the perceived usefulness of AGs. We can recommend that users with a limited technical background use ADTs or CORAS as a general graphical TM method. Further research on the acceptability of AGs to such an audience and the effect of a dedicated TM tool support is needed.