AutoMalDesc: Large-Scale Script Analysis for Cyber Threat Research

TL;DR

AutoMalDesc is an automated framework for large-scale static analysis of scripts that improves threat detection summaries through iterative learning, synthetic data, and comprehensive validation, aiding cybersecurity research.

Contribution

It introduces AutoMalDesc, a scalable, self-paced learning approach for script analysis that reduces manual annotation and enhances summary quality and classification accuracy.

Findings

Significant improvements in summary quality over iterations

Enhanced malware classification accuracy

Validated effectiveness through quantitative and qualitative assessments

Abstract

Generating thorough natural language explanations for threat detections remains an open problem in cybersecurity research, despite significant advances in automated malware detection systems. In this work, we present AutoMalDesc, an automated static analysis summarization framework that, following initial training on a small set of expert-curated examples, operates independently at scale. This approach leverages an iterative self-paced learning pipeline to progressively enhance output quality through synthetic data generation and validation cycles, eliminating the need for extensive manual data annotation. Evaluation across 3,600 diverse samples in five scripting languages demonstrates statistically significant improvements between iterations, showing consistent gains in both summary quality and classification accuracy. Our comprehensive validation approach combines quantitative metrics based on established malware labels with qualitative assessment from both human experts and LLM-based judges, confirming both technical precision and linguistic coherence of generated summaries. To facilitate reproducibility and advance research in this domain, we publish our complete dataset of more than 100K script samples, including annotated seed (0.9K) and test (3.6K) datasets, along with our methodology and evaluation framework.