RFCAudit: An LLM Agent for Functional Bug Detection in Network Protocols

TL;DR

RFCAudit is an LLM-based autonomous agent designed to detect functional bugs in network protocol implementations by checking conformance with RFC specifications, significantly aiding in ensuring protocol correctness and security.

Contribution

The paper introduces RFCAudit, a novel LLM-powered agent that automates semantic analysis for bug detection in network protocols, surpassing traditional static analysis methods.

Findings

Identified 47 bugs with 81.9% precision

20 bugs confirmed or fixed by developers

Effective semantic indexing and retrieval approach

Abstract

Functional correctness is critical for ensuring the reliability and security of network protocol implementations. Functional bugs, instances where implementations diverge from behaviors specified in RFC documents, can lead to severe consequences, including faulty routing, authentication bypasses, and service disruptions. Detecting these bugs requires deep semantic analysis across specification documents and source code, a task beyond the capabilities of traditional static analysis tools. This paper introduces RFCAudit, an autonomous agent that leverages large language models (LLMs) to detect functional bugs by checking conformance between network protocol implementations and their RFC specifications. Inspired by the human auditing procedure, RFCAudit comprises two key components: an indexing agent and a detection agent. The former hierarchically summarizes protocol code semantics, generating semantic indexes that enable the detection agent to narrow down the scanning scope. The latter employs demand-driven retrieval to iteratively collect additional relevant data structures and functions, eventually identifying potential inconsistencies with the RFC specifications effectively. We evaluate RFCAudit across six real-world network protocol implementations. RFCAudit identifies 47 functional bugs with 81.9% precision, of which 20 bugs have been confirmed or fixed by developers.