Towards Explainable Vulnerability Detection with Large Language Models

TL;DR

This paper introduces LLMVulExp, a framework that uses large language models to detect software vulnerabilities and generate detailed explanations, improving security analysis with high accuracy and interpretability.

Contribution

The paper presents a novel LLM-based approach for vulnerability detection and explanation, utilizing prompt techniques, instruction tuning with LoRA, and Chain-of-Thought strategies for enhanced performance.

Findings

Achieves over 90% F1 score on SeVC dataset

Provides detailed explanations including cause, location, and repair

Effectively combines detection accuracy with interpretability

Abstract

Software vulnerabilities pose significant risks to the security and integrity of software systems. Although prior studies have explored vulnerability detection using deep learning and pre-trained models, these approaches often fail to provide the detailed explanations necessary for developers to understand and remediate vulnerabilities effectively. The advent of large language models (LLMs) has introduced transformative potential due to their advanced generative capabilities and ability to comprehend complex contexts, offering new possibilities for addressing these challenges. In this paper, we propose LLMVulExp, an automated framework designed to specialize LLMs for the dual tasks of vulnerability detection and explanation. To address the challenges of acquiring high-quality annotated data and injecting domain-specific knowledge, LLMVulExp leverages prompt-based techniques for annotating vulnerability explanations and finetunes LLMs using instruction tuning with Low-Rank Adaptation (LoRA), enabling LLMVulExp to detect vulnerability types in code while generating detailed explanations, including the cause, location, and repair suggestions. Additionally, we employ a Chain-of-Thought (CoT) based key code extraction strategy to focus LLMs on analyzing vulnerability-prone code, further enhancing detection accuracy and explanatory depth. Our experimental results demonstrate that LLMVulExp achieves over a 90% F1 score on the SeVC dataset, effectively combining high detection accuracy with actionable and coherent explanations. This study highlights the feasibility of utilizing LLMs for real-world vulnerability detection and explanation tasks, providing critical insights into their adaptation and application in software security.