Early Detection of In-Memory Malicious Activity based on Run-time Environmental Features

TL;DR

This paper introduces a machine learning-based method for early detection of in-memory malicious activity on Windows systems, aiming to reduce false positives and overhead before malware exploitation occurs.

Contribution

It presents a novel end-to-end detection approach using runtime environment features, combining malware research, machine learning, and OS internals for improved pre-exploitation detection.

Findings

Reduced false positives and overhead in detection

Effective identification of malicious in-memory activity

Potential for extension against bypassing techniques

Abstract

In recent years malware has become increasingly sophisticated and difficult to detect prior to exploitation. While there are plenty of approaches to malware detection, they all have shortcomings when it comes to identifying malware correctly prior to exploitation. The trade-off is usually between false positives, causing overhead, preventing normal usage and the risk of letting the malware execute and cause damage to the target. We present a novel end-to-end solution for in-memory malicious activity detection done prior to exploitation by leveraging machine learning capabilities based on data from unique run-time logs, which are carefully curated in order to detect malicious activity in the memory of protected processes. This solution achieves reduced overhead and false positives as well as deployment simplicity. We implemented our solution for Windows-based systems, employing multi disciplinary knowledge from malware research, machine learning, and operating system internals. Our experimental evaluation yielded promising results. As we expect future sophisticated malware may try to bypass it, we also discuss how our solution can be extended to thwart such bypassing attempts.