VulBinLLM: LLM-powered Vulnerability Detection for Stripped Binaries

TL;DR

VulBinLLM introduces an LLM-based framework that enhances binary vulnerability detection through optimized decompilation and extended context reasoning, achieving state-of-the-art results on synthetic datasets.

Contribution

The paper presents VulBinLLM, a novel LLM-powered approach that improves binary vulnerability detection by integrating decompilation enhancements and advanced reasoning techniques.

Findings

Achieves state-of-the-art performance on Juliet dataset

Effectively detects vulnerabilities in stripped C/C++ binaries

Demonstrates the feasibility of LLMs in binary security analysis

Abstract

Recognizing vulnerabilities in stripped binary files presents a significant challenge in software security. Although some progress has been made in generating human-readable information from decompiled binary files with Large Language Models (LLMs), effectively and scalably detecting vulnerabilities within these binary files is still an open problem. This paper explores the novel application of LLMs to detect vulnerabilities within these binary files. We demonstrate the feasibility of identifying vulnerable programs through a combined approach of decompilation optimization to make the vulnerabilities more prominent and long-term memory for a larger context window, achieving state-of-the-art performance in binary vulnerability analysis. Our findings highlight the potential for LLMs to overcome the limitations of traditional analysis methods and advance the field of binary vulnerability detection, paving the way for more secure software systems. In this paper, we present Vul-BinLLM , an LLM-based framework for binary vulnerability detection that mirrors traditional binary analysis workflows with fine-grained optimizations in decompilation and vulnerability reasoning with an extended context. In the decompilation phase, Vul-BinLLM adds vulnerability and weakness comments without altering the code structure or functionality, providing more contextual information for vulnerability reasoning later. Then for vulnerability reasoning, Vul-BinLLM combines in-context learning and chain-of-thought prompting along with a memory management agent to enhance accuracy. Our evaluations encompass the commonly used synthetic dataset Juliet to evaluate the potential feasibility for analysis and vulnerability detection in C/C++ binaries. Our evaluations show that Vul-BinLLM is highly effective in detecting vulnerabilities on the compiled Juliet dataset.