Can Small GenAI Language Models Rival Large Language Models in Understanding Application Behavior?

TL;DR

This paper evaluates small and large GenAI language models in understanding application behavior, especially malware detection, showing small models are competitive in precision and recall while being more resource-efficient.

Contribution

It systematically compares small and large GenAI models in application behavior analysis, highlighting the practical viability of small models for resource-constrained environments.

Findings

Small models maintain competitive precision and recall.

Large models achieve higher overall accuracy.

Small models offer advantages in computational efficiency.

Abstract

Generative AI (GenAI) models, particularly large language models (LLMs), have transformed multiple domains, including natural language processing, software analysis, and code understanding. Their ability to analyze and generate code has enabled applications such as source code summarization, behavior analysis, and malware detection. In this study, we systematically evaluate the capabilities of both small and large GenAI language models in understanding application behavior, with a particular focus on malware detection as a representative task. While larger models generally achieve higher overall accuracy, our experiments show that small GenAI models maintain competitive precision and recall, offering substantial advantages in computational efficiency, faster inference, and deployment in resource-constrained environments. We provide a detailed comparison across metrics such as accuracy, precision, recall, and F1-score, highlighting each model's strengths, limitations, and operational feasibility. Our findings demonstrate that small GenAI models can effectively complement large ones, providing a practical balance between performance and resource efficiency in real-world application behavior analysis.