Improving the Identification of Real-world Malware's DNS Covert Channels Using Locality Sensitive Hashing

TL;DR

This paper introduces a novel method using Locality Sensitive Hashing and machine learning to detect and identify malware through DNS covert channels, improving accuracy and robustness over previous techniques.

Contribution

It is the first to apply Locality Sensitive Hashing for malware detection via DNS covert channels, enhancing identification accuracy and generalization to unseen malware.

Findings

Higher detection accuracy than prior methods

Reduced false positive rates

Effective classification of malware behaviors

Abstract

Nowadays, malware increasingly uses DNS-based covert channels in order to evade detection and maintain stealthy communication with its command-and-control servers. While prior work has focused on detecting such activity, identifying specific malware families and their behaviors from captured network traffic remains challenging due to the variability of DNS. In this paper, we present the first application of Locality Sensitive Hashing to the detection and identification of real-world malware utilizing DNS covert channels. Our approach encodes DNS subdomain sequences into statistical similarity features that effectively capture anomalies indicative of malicious activity. Combined with a Random Forest classifier, our method achieves higher accuracy and reduced false positive rates than prior approaches, while demonstrating improved robustness and generalization to previously unseen or modified malware samples. We further demonstrate that our approach enables reliable classification of malware behavior (e.g., uploading or downloading of files), based solely on DNS subdomains.