Can LLMs Deobfuscate Binary Code? A Systematic Analysis of Large Language Models into Pseudocode Deobfuscation

TL;DR

This paper introduces BinDeObfBench, a comprehensive benchmark for evaluating large language models' effectiveness in binary code deobfuscation, emphasizing reasoning and domain expertise over model size.

Contribution

It systematically assesses LLM-based binary deobfuscation across diverse obfuscation transformations and highlights the superiority of task-specific fine-tuning and reasoning strategies.

Findings

Deobfuscation performance relies more on reasoning than model scale.

Task-specific fine-tuning outperforms broad pre-training.

Reasoning models are robust across different ISAs and obfuscation levels.

Abstract

Deobfuscating binary code remains a fundamental challenge in reverse engineering, as obfuscation is widely used to hinder analysis and conceal program logic. Although large language models (LLMs) have shown promise in recovering semantics from obfuscated binaries, a systematic evaluation of their effectiveness is still lacking. In this work, we present BinDeObfBench, the first comprehensive benchmark for assessing LLM-based binary deobfuscation across diverse transformations spanning pre-compilation, compile-time, and post-compilation stages. Our evaluation shows that deobfuscation performance depends more on reasoning capability and domain expertise than on model scale, and that task-specific supervised fine-tuning consistently outperforms broad domain pre-training. Reasoning models can maintain robustness under severe obfuscation, generalize across different instruction set architectures (ISAs) and optimization levels. In-context learning benefits standard models but yields limited gains for reasoning models. Overall, our study highlights the importance of task-specific fine-tuning and reasoning-driven strategies, and positions BinDeObfBench as a basis for future work in binary deobfuscation.