Adapting Large Language Models to Emerging Cybersecurity using Retrieval Augmented Generation

TL;DR

This paper presents a retrieval-augmented generation framework that improves large language models' ability to adapt to emerging cybersecurity threats by enhancing their contextual understanding and reasoning capabilities.

Contribution

It introduces a novel RAG-based approach tailored for cybersecurity, demonstrating improved adaptability and reliability of LLMs in evolving threat landscapes.

Findings

Hybrid retrieval enhances LLM accuracy in cybersecurity tasks.

RAG improves temporal reasoning in threat detection.

Framework shows promise in real-world cybersecurity applications.

Abstract

Security applications are increasingly relying on large language models (LLMs) for cyber threat detection; however, their opaque reasoning often limits trust, particularly in decisions that require domain-specific cybersecurity knowledge. Because security threats evolve rapidly, LLMs must not only recall historical incidents but also adapt to emerging vulnerabilities and attack patterns. Retrieval-Augmented Generation (RAG) has demonstrated effectiveness in general LLM applications, but its potential for cybersecurity remains underexplored. In this work, we introduce a RAG-based framework designed to contextualize cybersecurity data and enhance LLM accuracy in knowledge retention and temporal reasoning. Using external datasets and the Llama-3-8B-Instruct model, we evaluate baseline RAG, an optimized hybrid retrieval approach, and conduct a comparative analysis across multiple performance metrics. Our findings highlight the promise of hybrid retrieval in strengthening the adaptability and reliability of LLMs for cybersecurity tasks.