Can Large Language Models Understand Intermediate Representations in Compilers?

TL;DR

This study evaluates six state-of-the-art large language models' ability to understand compiler intermediate representations, revealing strengths in syntax parsing but challenges in instruction-level reasoning and control flow comprehension.

Contribution

The paper provides the first comprehensive empirical assessment of LLMs on IR understanding tasks, highlighting specific weaknesses and proposing targeted improvements.

Findings

LLMs can parse IR syntax and identify high-level structures.

Models struggle with control flow and instruction-level reasoning.

Fine-tuning and IR-specific architectures can enhance performance.

Abstract

Intermediate Representations (IRs) play a critical role in compiler design and program analysis, yet their comprehension by Large Language Models (LLMs) remains underexplored. In this paper, we present an explorative empirical study evaluating the capabilities of six state-of-the-art LLMs: GPT-4, GPT-3, DeepSeek, Gemma 2, Llama 3, and Code Llama, in understanding IRs. Specifically, we assess model performance across four core tasks: control flow graph reconstruction, decompilation, code summarization, and execution reasoning. While LLMs exhibit competence in parsing IR syntax and identifying high-level structures, they consistently struggle with instruction-level reasoning, especially in control flow reasoning, loop handling, and dynamic execution. Common failure modes include misinterpreting branching instructions, omitting critical operations, and relying on heuristic reasoning rather than precise instruction-level logic. Our findings highlight the need for IR-specific enhancements in LLM design. We recommend fine-tuning on structured IR datasets and integrating control-flow-sensitive architectures to improve model effectiveness. All experimental data and source code are publicly available at