Nested Dirichlet models for unsupervised attack pattern detection in honeypot data

TL;DR

This paper introduces advanced Dirichlet-based models for unsupervised clustering of honeypot attack data, enabling threat-hunting experts to identify attack groups and outliers more effectively, including novel malware variants.

Contribution

It develops hierarchical and nonparametric Dirichlet models tailored for command-line attack data, enhancing interpretability and detection of unknown attack types.

Findings

Successfully identified a new MIRAI variant targeting cryptocurrency infrastructure.

Improved attack clustering interpretability through hierarchical topic structures.

Demonstrated effectiveness of Bayesian nonparametric models in uncovering novel threats.

Abstract

Cyber-systems are under near-constant threat from intrusion attempts. Attacks types vary, but each attempt typically has a specific underlying intent, and the perpetrators are typically groups of individuals with similar objectives. Clustering attacks appearing to share a common intent is very valuable to threat-hunting experts. This article explores Dirichlet distribution topic models for clustering terminal session commands collected from honeypots, which are special network hosts designed to entice malicious attackers. The main practical implications of clustering the sessions are two-fold: finding similar groups of attacks, and identifying outliers. A range of statistical models are considered, adapted to the structures of command-line syntax. In particular, concepts of primary and secondary topics, and then session-level and command-level topics, are introduced into the models to improve interpretability. The proposed methods are further extended in a Bayesian nonparametric fashion to allow unboundedness in the vocabulary size and the number of latent intents. The methods are shown to discover an unusual MIRAI variant which attempts to take over existing cryptocurrency coin-mining infrastructure, not detected by traditional topic-modelling approaches.