Feedback-Driven Execution for LLM-Based Binary Analysis

TL;DR

FORGE introduces a feedback-driven, decomposed execution system for LLM-based binary analysis, enabling scalable, long-horizon reasoning and improved vulnerability detection in complex firmware binaries.

Contribution

It presents FORGE, a novel feedback-driven execution framework with a Dynamic Forest of Agents to enhance LLM-based binary analysis.

Findings

FORGE identified 1,274 vulnerabilities in 591 binaries.

Achieved 72.3% precision in vulnerability detection.

Broader vulnerability coverage than prior methods.

Abstract

Binary analysis increasingly relies on large language models (LLMs) to perform semantic reasoning over complex program behaviors. However, existing approaches largely adopt a one-pass execution paradigm, where reasoning operates over a fixed program representation constructed by static analysis tools. This formulation limits the ability to adapt exploration based on intermediate results and makes it difficult to sustain long-horizon, multi-path analysis under constrained context. We present FORGE, a system that rethinks LLM-based analysis as a feedback-driven execution process. FORGE interleaves reasoning and tool interaction through a reasoning-action-observation loop, enabling incremental exploration and evidence construction. To address the instability of long-horizon reasoning, we introduce a Dynamic Forest of Agents (FoA), a decomposed execution model that dynamically coordinates parallel exploration while bounding per-agent context. We evaluate FORGE on 3,457 real-world firmware binaries. FORGE identifies 1,274 vulnerabilities across 591 unique binaries, achieving 72.3% precision while covering a broader range of vulnerability types than prior approaches. These results demonstrate that structuring LLM-based analysis as a decomposed, feedback-driven execution system enables both scalable reasoning and high-quality outcomes in long-horizon tasks.