RAVEN: Retrieval-Augmented Vulnerability Exploration Network for Memory Corruption Analysis in User Code and Binary Programs

TL;DR

RAVEN is a framework that uses large language models and retrieval-augmented generation to automatically produce detailed vulnerability analysis reports from source code, enhancing cybersecurity documentation.

Contribution

The paper introduces RAVEN, a novel LLM-based system with retrieval and evaluation modules for automated vulnerability report generation, addressing a gap in cybersecurity automation.

Findings

RAVEN achieved an average report quality score of 54.21%.

The framework effectively covers 15 CWE vulnerability types.

Evaluation on 105 samples demonstrates promising results for automated documentation.

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities across various cybersecurity tasks, including vulnerability classification, detection, and patching. However, their potential in automated vulnerability report documentation and analysis remains underexplored. We present RAVEN (Retrieval Augmented Vulnerability Exploration Network), a framework leveraging LLM agents and Retrieval Augmented Generation (RAG) to synthesize comprehensive vulnerability analysis reports. Given vulnerable source code, RAVEN generates reports following the Google Project Zero Root Cause Analysis template. The framework uses four modules: an Explorer agent for vulnerability identification, a RAG engine retrieving relevant knowledge from curated databases including Google Project Zero reports and CWE entries, an Analyst agent for impact and exploitation assessment, and a Reporter agent for structured report generation. To ensure quality, RAVEN includes a task specific LLM Judge evaluating reports across structural integrity, ground truth alignment, code reasoning quality, and remediation quality. We evaluate RAVEN on 105 vulnerable code samples covering 15 CWE types from the NIST-SARD dataset. Results show an average quality score of 54.21%, supporting the effectiveness of our approach for automated vulnerability documentation.