Holmes: An Efficient and Lightweight Semantic Based Anomalous Email Detector

TL;DR

Holmes is a lightweight semantic engine that converts emails into sentences using word embedding to detect anomalies and malicious threats that traditional signature-based methods often miss, enhancing enterprise email security.

Contribution

This paper introduces Holmes, a novel semantic-based email anomaly detection system that effectively identifies unknown and concealed malicious emails in real-world enterprise environments.

Findings

Holmes detects threats beyond traditional anti-spam capabilities.

It discovers more concealed malicious emails immune to commercial tools.

High detection capability in real-world enterprise settings.

Abstract

Email threat is a serious issue for enterprise security. The threat can be in various malicious forms, such as phishing, fraud, blackmail and malvertisement. The traditional anti-spam gateway often maintains a greylist to filter out unexpected emails based on suspicious vocabularies present in the email's subject and contents. However, this type of signature-based approach cannot effectively discover novel and unknown suspicious emails that utilize various evolving malicious payloads. To address the problem, in this paper, we present Holmes, an efficient and lightweight semantic based engine for anomalous email detection. Holmes can convert each email event log into a sentence through word embedding and then identify abnormalities that deviate from a historical baseline based on those translated sentences. We have evaluated the performance of Holmes in a real-world enterprise environment, where around 5,000 emails are sent/received each day. In our experiments, Holmes shows a high capability to detect email threats, especially those that cannot be handled by the enterprise anti-spam gateway. It is also demonstrated through our experiment that Holmes can discover more concealed malicious emails that are immune from several commercial detection tools.