An Attempt to Catch Up with JIT Compilers: The False Lead of Optimizing Inline Caches

TL;DR

This paper investigates whether adding Dynamic Binary Modification to Ahead-of-Time JavaScript compilers improves performance, finding that reducing memory accesses does not necessarily lead to faster execution on modern architectures.

Contribution

It introduces a new DBM-based optimization for inline caches in AoT compilers and evaluates its impact, revealing limitations of such optimizations on current hardware.

Findings

DBM optimization does not reduce execution time on x86_64.

Reducing memory accesses alone is insufficient for speedup on modern CPUs.

Sophisticated compiler optimizations are only beneficial if hardware cannot automatically optimize code.

Abstract

Context: Just-in-Time (JIT) compilers are able to specialize the code they generate according to a continuous profiling of the running programs. This gives them an advantage when compared to Ahead-of-Time (AoT) compilers that must choose the code to generate once for all. Inquiry: Is it possible to improve the performance of AoT compilers by adding Dynamic Binary Modification (DBM) to the executions? Approach: We added to the Hopc AoT JavaScript compiler a new optimization based on DBM to the inline cache (IC), a classical optimization dynamic languages use to implement object property accesses efficiently. Knowledge: Reducing the number of memory accesses as the new optimization does, does not shorten execution times on contemporary architectures. Grounding: The DBM optimization we have implemented is fully operational on x86_64 architectures. We have conducted several experiments to evaluate its impact on performance and to study the reasons of the lack of acceleration. Importance: The (negative) result we present in this paper sheds new light on the best strategy to be used to implement dynamic languages. It tells that the old days were removing instructions or removing memory reads always yielded to speed up is over. Nowadays, implementing sophisticated compiler optimizations is only worth the effort if the processor is not able by itself to accelerate the code. This result applies to AoT compilers as well as JIT compilers.