A Formal Semantics of the GraalVM Intermediate Representation

TL;DR

This paper defines a formal semantics for GraalVM's intermediate representation using Isabelle/HOL, enabling verification of optimization passes through rigorous proofs of canonicalization and conditional elimination transformations.

Contribution

It introduces a formal, executable semantics for GraalVM IR in Isabelle/HOL, facilitating verification of compiler optimizations.

Findings

Proved correctness of canonicalization optimizations.

Verified conditional elimination optimizations.

Established a formal foundation for GraalVM IR semantics.

Abstract

The optimization phase of a compiler is responsible for transforming an intermediate representation (IR) of a program into a more efficient form. Modern optimizers, such as that used in the GraalVM compiler, use an IR consisting of a sophisticated graph data structure that combines data flow and control flow into the one structure. As part of a wider project on the verification of optimization passes of GraalVM, this paper describes a semantics for its IR within Isabelle/HOL. The semantics consists of a big-step operational semantics for data nodes (which are represented in a graph-based static single assignment (SSA) form) and a small-step operational semantics for handling control flow including heap-based reads and writes, exceptions, and method calls. We have proved a suite of canonicalization optimizations and conditional elimination optimizations with respect to the semantics.