A Comprehensive Study on Learning-Based PE Malware Family Classification Methods

TL;DR

This paper provides a comprehensive empirical comparison of learning-based PE malware family classification methods, highlighting their performance, challenges with concept drift, and industry adoption barriers.

Contribution

It offers the first thorough comparison of image-based, binary-based, and disassembly-based approaches across multiple datasets and discusses practical industry considerations.

Findings

No method class significantly outperforms others.

All methods degrade under concept drift, with an average F1-score drop of 32.23%.

High prediction time and memory use hinder industry adoption.

Abstract

Driven by the high profit, Portable Executable (PE) malware has been consistently evolving in terms of both volume and sophistication. PE malware family classification has gained great attention and a large number of approaches have been proposed. With the rapid development of machine learning techniques and the exciting results they achieved on various tasks, machine learning algorithms have also gained popularity in the PE malware family classification task. Three mainstream approaches that use learning based algorithms, as categorized by the input format the methods take, are image-based, binary-based and disassembly-based approaches. Although a large number of approaches are published, there is no consistent comparisons on those approaches, especially from the practical industry adoption perspective. Moreover, there is no comparison in the scenario of concept drift, which is a fact for the malware classification task due to the fast evolving nature of malware. In this work, we conduct a thorough empirical study on learning-based PE malware classification approaches on 4 different datasets and consistent experiment settings. Based on the experiment results and an interview with our industry partners, we find that (1) there is no individual class of methods that significantly outperforms the others; (2) All classes of methods show performance degradation on concept drift (by an average F1-score of 32.23%); and (3) the prediction time and high memory consumption hinder existing approaches from being adopted for industry usage.