LoRA-based Parameter-Efficient LLMs for Continuous Learning in Edge-based Malware Detection

TL;DR

This paper introduces a LoRA-based continual learning framework for edge malware detection using lightweight transformers, enabling cross-device knowledge sharing and improved accuracy under resource constraints.

Contribution

It proposes a novel edge malware detection architecture combining local incremental training with global LoRA adapter sharing for domain adaptation.

Findings

Achieves 20-25% accuracy improvement on unseen attacks.

Maintains stable loss and F1 scores across multiple learning rounds.

LoRA modules add less than 1% to model size, suitable for edge devices.

Abstract

The proliferation of edge devices has created an urgent need for security solutions capable of detecting malware in real time while operating under strict computational and memory constraints. Recently, Large Language Models (LLMs) have demonstrated remarkable capabilities in recognizing complex patterns, yet their deployment on edge devices remains impractical due to their resource demands. However, in edge malware detection, static or centrally retrained models degrade under evolving threats and heterogeneous traffic; locally trained models become siloed and fail to transfer across domains. To overcome these limitations, in this paper, we present a continuous learning architecture for edge-based malware detection that combines local adaptation on each device with global knowledge sharing through parameter-efficient LoRA adapters. Lightweight transformer models (DistilBERT, DistilGPT-2, TinyT5) run on edge nodes and are incrementally fine-tuned on device-specific traffic; only the resulting LoRA modules are aggregated by a lightweight coordinator and redistributed, enabling cross-device generalization without exchanging raw data. We evaluate on two public IoT security datasets, Edge-IIoTset and TON-IoT, under multi-round learning to simulate evolving threats. Compared to isolated fine-tuning, the LoRA-based exchange yields up to 20-25% accuracy gains when models encounter previously unseen attacks from another domain, while maintaining stable loss and F1 across rounds. LoRA adds less than 1% to model size (~0.6-1.8 MB), making updates practical for constrained edge hardware.