Beyond the Wrapper: Identifying Artifact Reliance in Static Malware Classifiers using TRUSTEE

TL;DR

This paper introduces a framework using TRUSTEE to identify artifact reliance in static malware classifiers, revealing their sensitivity to dataset biases and artifacts like packing rather than true malicious behavior.

Contribution

The authors propose a novel two-part interpretability framework to diagnose and understand artifact reliance in static malware classifiers, improving model robustness.

Findings

Top features are packing artifacts and PE metadata, not malicious semantics.

Malware classifiers are highly sensitive to dataset composition.

The framework enables reproducible diagnosis of classifier biases.

Abstract

Modern cybersecurity relies heavily on static machine-learning-based malware classifiers. However, transformations such as packing and other non-semantic modifications applied to executable files limit their reliability. Malware classifiers often learn these unnecessary artifacts rather than the true binary behavior because of the high association between maliciousness and packing. Moreover, these malware classifiers are black boxes, making it difficult to understand what they learn. To address this issue, we proposed a two-part framework using the post-hoc interpretability XAI tool TRUSTEE, followed by a manual analysis of the top features. We conducted several controlled experiments by varying the dataset composition ratios to understand their impact on the results. The top-ranked features across all experiments, identified by TRUSTEE, were predominantly packing artifacts, portable executable(PE) metadata, and n-grams at the string level, rather than malicious semantics. These results suggest that these malware classifiers are highly sensitive to dataset composition and can misinterpret packing as malicious behavior. Our proposed framework allows for the reproducible diagnosis of such biases and forms a guideline for building more robust and semantically meaningful malware detection models