Exploring the Efficacy of Large Language Models (GPT-4) in Binary Reverse Engineering

TL;DR

This paper evaluates GPT-4's ability to perform binary reverse engineering, demonstrating its strengths in general code understanding and highlighting current limitations in complex security analysis tasks.

Contribution

It provides a structured experimental assessment of GPT-4's capabilities in binary reverse engineering, offering insights into its potential and areas for improvement.

Findings

GPT-4 shows proficiency in interpreting general code.

Effectiveness varies in malware and security analysis.

Methodologies for evaluating LLMs in RE are proposed.

Abstract

This study investigates the capabilities of Large Language Models (LLMs), specifically GPT-4, in the context of Binary Reverse Engineering (RE). Employing a structured experimental approach, we analyzed the LLM's performance in interpreting and explaining human-written and decompiled codes. The research encompassed two phases: the first on basic code interpretation and the second on more complex malware analysis. Key findings indicate LLMs' proficiency in general code understanding, with varying effectiveness in detailed technical and security analyses. The study underscores the potential and current limitations of LLMs in reverse engineering, revealing crucial insights for future applications and improvements. Also, we examined our experimental methodologies, such as methods of evaluation and data constraints, which provided us with a technical vision for any future research activity in this field.