Supporting On-Stack Replacement in Unstructured Languages by Loop Reconstruction and Extraction

TL;DR

This paper introduces a method to enable on-stack replacement in unstructured language bytecode interpreters by reconstructing loops, demonstrated with a prototype that significantly speeds up program warm-up times.

Contribution

It proposes a novel approach to reconstruct loops in unstructured bytecode to support OSR, implemented in Sulong for LLVM bitcode, improving warm-up performance.

Findings

Up to 9x speed-up in warm-up times for certain benchmarks

Effective loop reconstruction in unstructured bytecode interpreters

Generalizable approach for various bytecode interpreters

Abstract

On-stack replacement (OSR) is a common technique employed by dynamic compilers to reduce program warm-up time. OSR allows switching from interpreted to compiled code during the execution of this code. The main targets are long running loops, which need to be represented explicitly, with dedicated information about condition and body, to be optimized at run time. Bytecode interpreters, however, represent control flow implicitly via unstructured jumps and thus do not exhibit the required high-level loop representation. To enable OSR also for jump-based - often called unstructured - languages, we propose the partial reconstruction of loops in order to explicitly represent them in a bytecode interpreter. Besides an outline of the general idea, we implemented our approach in Sulong, a bytecode interpreter for LLVM bitcode, which allows the execution of C/C++. We conducted an evaluation with a set of C benchmarks, which showed speed-ups in warm-up of up to 9x for certain benchmarks. This facilitates execution of programs with long-running loops in rarely called functions, which would yield significant slowdown without OSR. While shown with a prototype implementation, the overall idea of our approach is generalizable for all bytecode interpreters.