Malware Detection based on API Calls: A Reproducibility Study

TL;DR

This paper reproduces and validates a malware detection method based on API call frequency analysis, confirming its effectiveness and robustness with consistent results across multiple experiments.

Contribution

It independently reproduces the original API call-based malware detection approach, confirming its high accuracy, reproducibility, and practical viability.

Findings

F1-scores exceeded original results by up to 2.57%.

High reproducibility with standard deviations below 0.5%.

Unigram model proved effective as a lightweight detector.

Abstract

This study independently reproduces the malware detection methodology presented by Felli cious et al. [7], which employs order-invariant API call frequency analysis using Random Forest classification. We utilized the original public dataset (250,533 training samples, 83,511 test samples) and replicated four model variants: Unigram, Bigram, Trigram, and Combined n gram approaches. Our reproduction successfully validated all key findings, achieving F1-scores that exceeded the original results by 0.99% to 2.57% across all models at the optimal API call length of 2,500. The Unigram model achieved F1=0.8717 (original: 0.8631), confirming its ef fectiveness as a lightweight malware detector. Across three independent experimental runs with different random seeds, we observed remarkably consistent results with standard deviations be low 0.5%, demonstrating high reproducibility. This study validates the robustness and scientific rigor of the original methodology while confirming the practical viability of frequency-based API call analysis for malware detection.