Phishing Email Detection Using Inputs From Artificial Intelligence

TL;DR

This paper explores using artificial intelligence and natural language processing to detect phishing emails by analyzing signals similar to those taught in security training, aiming to improve automatic detection and training methods.

Contribution

It introduces a new dataset with annotated signals for phishing detection and develops baseline models, bridging human training signals with machine learning approaches.

Findings

Baseline models perform comparably to human annotators on signal detection.

Analysis provides insights for improving training curricula for both humans and machines.

The dataset enables further research in AI-driven phishing detection.

Abstract

Enterprise security is increasingly being threatened by social engineering attacks, such as phishing, which deceive employees into giving access to enterprise data. To protect both the users themselves and enterprise data, more and more organizations provide cyber security training that seeks to teach employees/customers to identify and report suspicious content. By its very nature, such training seeks to focus on signals that are likely to persist across a wide range of attacks. Further, it expects the user to apply the learnings from these training on e-mail messages that were not filtered by existing, automatic enterprise security (e.g., spam filters and commercial phishing detection software). However, relying on such training now shifts the detection of phishing from an automatic process to a human driven one which is fallible especially when a user errs due to distraction, forgetfulness, etc. In this work we explore treating this type of detection as a natural language processing task and modifying training pipelines accordingly. We present a dataset with annotated labels where these labels are created from the classes of signals that users are typically asked to identify in such training. We also present baseline classifier models trained on these classes of labels. With a comparative analysis of performance between human annotators and the models on these labels, we provide insights which can contribute to the improvement of the respective curricula for both machine and human training.