Toward Explainable Users: Using NLP to Enable AI to Understand Users' Perceptions of Cyber Attacks

TL;DR

This paper explores how NLP techniques can interpret users' perceptions of cyber attack consequences, revealing that users mainly focus on keywords rather than semantics, and introduces AI as a tool to explain user behavior.

Contribution

It is the first study to use AI techniques to explain and model users' perceptions and mental models regarding cyber security threats.

Findings

Participants relied on keyword matching over semantics in their perceptions.

NLP methods can reveal users' focus on specific keywords.

The study demonstrates AI's potential in explaining user perceptions.

Abstract

To understand how end-users conceptualize consequences of cyber security attacks, we performed a card sorting study, a well-known technique in Cognitive Sciences, where participants were free to group the given consequences of chosen cyber attacks into as many categories as they wished using rationales they see fit. The results of the open card sorting study showed a large amount of inter-participant variation making the research team wonder how the consequences of security attacks were comprehended by the participants. As an exploration of whether it is possible to explain user's mental model and behavior through Artificial Intelligence (AI) techniques, the research team compared the card sorting data with the outputs of a number of Natural Language Processing (NLP) techniques with the goal of understanding how participants perceived and interpreted the consequences of cyber attacks written in natural languages. The results of the NLP-based exploration methods revealed an interesting observation implying that participants had mostly employed checking individual keywords in each sentence to group cyber attack consequences together and less considered the semantics behind the description of consequences of cyber attacks. The results reported in this paper are seemingly useful and important for cyber attacks comprehension from user's perspectives. To the best of our knowledge, this paper is the first introducing the use of AI techniques in explaining and modeling users' behavior and their perceptions about a context. The novel idea introduced here is about explaining users using AI.