Explaining Software Vulnerabilities with Large Language Models

TL;DR

This paper introduces SAFE, an IDE plugin utilizing GPT-4 to explain security vulnerabilities detected by SAST tools, aiming to improve developer understanding and usability.

Contribution

The paper presents a novel hybrid approach using LLMs to enhance explainability of SAST tool warnings within IDEs, addressing usability limitations.

Findings

SAFE significantly helps developers understand vulnerabilities

Explanations improve the usability of SAST tools

Study shows positive impact on beginner to intermediate developers

Abstract

The prevalence of security vulnerabilities has prompted companies to adopt static application security testing (SAST) tools for vulnerability detection. Nevertheless, these tools frequently exhibit usability limitations, as their generic warning messages do not sufficiently communicate important information to developers, resulting in misunderstandings or oversight of critical findings. In light of recent developments in Large Language Models (LLMs) and their text generation capabilities, our work investigates a hybrid approach that uses LLMs to tackle the SAST explainability challenges. In this paper, we present SAFE, an Integrated Development Environment (IDE) plugin that leverages GPT-4o to explain the causes, impacts, and mitigation strategies of vulnerabilities detected by SAST tools. Our expert user study findings indicate that the explanations generated by SAFE can significantly assist beginner to intermediate developers in understanding and addressing security vulnerabilities, thereby improving the overall usability of SAST tools.