ChatNVD: Advancing Cybersecurity Vulnerability Assessment with Large Language Models

TL;DR

ChatNVD leverages large language models to generate accessible summaries of cybersecurity vulnerabilities, improving understanding and assessment accuracy, with GPT-4o Mini showing superior performance in identifying and explaining vulnerabilities.

Contribution

Introduces ChatNVD, a novel LLM-powered tool that enhances cybersecurity vulnerability summaries using multiple LLM variants, with a focus on practical performance evaluation.

Findings

GPT-4o Mini achieves over 92% accuracy

GPT-4o Mini has the lowest error rates

GPT-4o Mini is most reliable for real-world use

Abstract

The increasing frequency and sophistication of cybersecurity vulnerabilities in software systems underscores the need for more robust and effective vulnerability assessment methods. However, existing approaches often rely on highly technical and abstract frameworks, which hinder understanding and increase the likelihood of exploitation, resulting in severe cyberattacks. In this paper, we introduce ChatNVD, a support tool powered by Large Language Models (LLMs) that leverages the National Vulnerability Database (NVD) to generate accessible, context-rich summaries of software vulnerabilities. We develop three variants of ChatNVD, utilizing three prominent LLMs: GPT-4o Mini by OpenAI, LLaMA 3 by Meta, and Gemini 1.5 Pro by Google. To evaluate their performance, we conduct a comparative evaluation focused on their ability to identify, interpret, and explain software vulnerabilities. Our results demonstrate that GPT-4o Mini outperforms the other models, achieving over 92% accuracy and the lowest error rates, making it the most reliable option for real-world vulnerability assessment.