LLM-FS: Zero-Shot Feature Selection for Effective and Interpretable Malware Detection

TL;DR

This paper explores using large language models in a zero-shot setting to guide feature selection for malware detection, achieving competitive results with enhanced interpretability and stability compared to traditional methods.

Contribution

It introduces a novel zero-shot LLM-guided feature selection approach for malware detection, emphasizing interpretability and performance without relying on labeled data.

Findings

LLM-guided FS achieves competitive accuracy and metrics.

Zero-shot LLM FS offers improved interpretability and stability.

Reduces dependence on labeled data for feature selection.

Abstract

Feature selection (FS) remains essential for building accurate and interpretable detection models, particularly in high-dimensional malware datasets. Conventional FS methods such as Extra Trees, Variance Threshold, Tree-based models, Chi-Squared tests, ANOVA, Random Selection, and Sequential Attention rely primarily on statistical heuristics or model-driven importance scores, often overlooking the semantic context of features. Motivated by recent progress in LLM-driven FS, we investigate whether large language models (LLMs) can guide feature selection in a zero-shot setting, using only feature names and task descriptions, as a viable alternative to traditional approaches. We evaluate multiple LLMs (GPT-5.0, GPT-4.0, Gemini-2.5 etc.) on the EMBOD dataset (a fusion of EMBER and BODMAS benchmark datasets), comparing them against established FS methods across several classifiers, including Random Forest, Extra Trees, MLP, and KNN. Performance is assessed using accuracy, precision, recall, F1, AUC, MCC, and runtime. Our results demonstrate that LLM-guided zero-shot feature selection achieves competitive performance with traditional FS methods while offering additional advantages in interpretability, stability, and reduced dependence on labeled data. These findings position zero-shot LLM-based FS as a promising alternative strategy for effective and interpretable malware detection, paving the way for knowledge-guided feature selection in security-critical applications