Obfuscated Malware Detection: Investigating Real-world Scenarios through Memory Analysis

TL;DR

This paper investigates the use of machine learning algorithms applied to memory dump analysis for detecting obfuscated malware in real-world scenarios, aiming to improve cybersecurity defenses against sophisticated threats.

Contribution

It introduces a cost-effective memory-based malware detection system evaluated on the CIC-MalMem-2022 dataset, comparing various machine learning algorithms' effectiveness.

Findings

Decision trees perform well in malware detection.

Ensemble methods improve detection accuracy.

Neural networks show potential but require further tuning.

Abstract

In the era of the internet and smart devices, the detection of malware has become crucial for system security. Malware authors increasingly employ obfuscation techniques to evade advanced security solutions, making it challenging to detect and eliminate threats. Obfuscated malware, adept at hiding itself, poses a significant risk to various platforms, including computers, mobile devices, and IoT devices. Conventional methods like heuristic-based or signature-based systems struggle against this type of malware, as it leaves no discernible traces on the system. In this research, we propose a simple and cost-effective obfuscated malware detection system through memory dump analysis, utilizing diverse machine-learning algorithms. The study focuses on the CIC-MalMem-2022 dataset, designed to simulate real-world scenarios and assess memory-based obfuscated malware detection. We evaluate the effectiveness of machine learning algorithms, such as decision trees, ensemble methods, and neural networks, in detecting obfuscated malware within memory dumps. Our analysis spans multiple malware categories, providing insights into algorithmic strengths and limitations. By offering a comprehensive assessment of machine learning algorithms for obfuscated malware detection through memory analysis, this paper contributes to ongoing efforts to enhance cybersecurity and fortify digital ecosystems against evolving and sophisticated malware threats. The source code is made open-access for reproducibility and future research endeavours. It can be accessed at https://bit.ly/MalMemCode.