Deep Learning for Unsupervised Insider Threat Detection in Structured Cybersecurity Data Streams

TL;DR

This paper introduces an online unsupervised deep learning method for real-time insider threat detection in cybersecurity logs, improving interpretability and outperforming traditional anomaly detection techniques.

Contribution

The paper presents a novel deep recurrent neural network approach for unsupervised insider threat detection that enhances interpretability and detection performance in streaming cybersecurity data.

Findings

Deep models outperform PCA, SVM, and Isolation Forest baselines.

Insider threat events scored in the 95.53 percentile on average.

Approach reduces analyst workload significantly.

Abstract

Analysis of an organization's computer network activity is a key component of early detection and mitigation of insider threat, a growing concern for many organizations. Raw system logs are a prototypical example of streaming data that can quickly scale beyond the cognitive power of a human analyst. As a prospective filter for the human analyst, we present an online unsupervised deep learning approach to detect anomalous network activity from system logs in real time. Our models decompose anomaly scores into the contributions of individual user behavior features for increased interpretability to aid analysts reviewing potential cases of insider threat. Using the CERT Insider Threat Dataset v6.2 and threat detection recall as our performance metric, our novel deep and recurrent neural network models outperform Principal Component Analysis, Support Vector Machine and Isolation Forest based anomaly detection baselines. For our best model, the events labeled as insider threat activity in our dataset had an average anomaly score in the 95.53 percentile, demonstrating our approach's potential to greatly reduce analyst workloads.