MORTON: Detection of Malicious Routines in Large-Scale DNS Traffic

TL;DR

MORTON is a scalable, robust method that detects malicious routines in enterprise DNS traffic using signal processing and neural networks, outperforming existing methods especially against sophisticated malware communications.

Contribution

Introduces MORTON, a novel scalable approach combining signal processing and neural networks for detecting malicious DNS routines in large enterprise networks.

Findings

MORTON achieves comparable accuracy to existing methods in synthetic tests.

It outperforms others in detecting complex bot communication techniques.

Demonstrates effectiveness and superiority in real-world enterprise DNS traffic.

Abstract

In this paper, we present MORTON, a method that identifies compromised devices in enterprise networks based on the existence of routine DNS communication between devices and disreputable host names. With its compact representation of the input data and use of efficient signal processing and a neural network for classification, MORTON is designed to be accurate, robust, and scalable. We evaluate MORTON using a large dataset of corporate DNS logs and compare it with two recently proposed beaconing detection methods aimed at detecting malware communication. The results demonstrate that while MORTON's accuracy in a synthetic experiment is comparable to that of the other methods, it outperforms those methods in terms of its ability to detect sophisticated bot communication techniques, such as multistage channels, as well as in its robustness and efficiency. In a real-world evaluation, which includes previously unreported threats, MORTON and the two compared methods were deployed to monitor the (unlabeled) DNS traffic of two global enterprises for a week-long period; this evaluation demonstrates the effectiveness of MORTON in real-world scenarios and showcases its superiority in terms of true and false positive rates.