End-To-End Anomaly Detection for Identifying Malicious Cyber Behavior through NLP-Based Log Embeddings

TL;DR

This paper introduces an end-to-end neural framework utilizing NLP-inspired log embeddings to improve anomaly detection of malicious cyber activities, demonstrating effectiveness on DARPA OpTC data.

Contribution

The work presents a novel deep learning approach that eliminates ad-hoc feature engineering by directly learning from log data using NLP techniques.

Findings

Effective detection of malicious cyber behaviors

Outperforms traditional rule-based IDS

Validated on DARPA OpTC dataset

Abstract

Rule-based IDS (intrusion detection systems) are being replaced by more robust neural IDS, which demonstrate great potential in the field of Cybersecurity. However, these ML approaches continue to rely on ad-hoc feature engineering techniques, which lack the capacity to vectorize inputs in ways that are fully relevant to the discovery of anomalous cyber activity. We propose a deep end-to-end framework with NLP-inspired components for identifying potentially malicious behaviors on enterprise computer networks. We also demonstrate the efficacy of this technique on the recently released DARPA OpTC data set.