Rethinking On-Device LLM Reasoning: Why Analogical Mapping Outperforms Abstract Thinking for IoT DDoS Detection

TL;DR

This paper proposes a novel IoT DDoS detection framework using on-device LLMs that combines Chain-of-Thought reasoning with retrieval-augmented generation, significantly improving detection accuracy under resource constraints.

Contribution

It introduces a new detection approach integrating CoT and RAG for small ODLLMs, demonstrating enhanced performance in IoT security scenarios.

Findings

Achieved macro-average F1 scores up to 0.85 with few-shot prompting.

Exemplar-based reasoning significantly improves classification accuracy.

CoT and RAG methods outperform traditional prompting strategies.

Abstract

The rapid expansion of IoT deployments has intensified cybersecurity threats, notably Distributed Denial of Service (DDoS) attacks, characterized by increasingly sophisticated patterns. Leveraging Generative AI through On-Device Large Language Models (ODLLMs) provides a viable solution for real-time threat detection at the network edge, though limited computational resources present challenges for smaller ODLLMs. This paper introduces a novel detection framework that integrates Chain-of-Thought (CoT) reasoning with Retrieval-Augmented Generation (RAG), tailored specifically for IoT edge environments. We systematically evaluate compact ODLLMs, including LLaMA 3.2 (1B, 3B) and Gemma 3 (1B, 4B), using structured prompting and exemplar-driven reasoning strategies. Experimental results demonstrate substantial performance improvements with few-shot prompting, achieving macro-average F1 scores as high as 0.85. Our findings highlight the significant advantages of incorporating exemplar-based reasoning, underscoring that CoT and RAG approaches markedly enhance small ODLLMs' capabilities in accurately classifying complex network attacks under stringent resource constraints.