AnICA: Analyzing Inconsistencies in Microarchitectural Code Analyzers

TL;DR

AnICA is a systematic tool that analyzes and explains inconsistencies among microarchitectural code analyzers, revealing bugs and assumptions in tools like llvm-mca and AMD Zen microarchitectures.

Contribution

It introduces AnICA, a novel approach combining differential testing and abstract interpretation to identify and characterize inconsistencies in code analyzers.

Findings

AnICA summarizes thousands of inconsistencies efficiently.

It automatically detects and characterizes known and unknown bugs.

Reveals microarchitectural quirks in AMD Zen CPUs.

Abstract

Microarchitectural code analyzers, i.e., tools that estimate the throughput of machine code basic blocks, are important utensils in the tool belt of performance engineers. Recent tools like llvm-mca, uiCA, and Ithemal use a variety of techniques and different models for their throughput predictions. When put to the test, it is common to see these state-of-the-art tools give very different results. These inconsistencies are either errors, or they point to different and rarely documented assumptions made by the tool designers. In this paper, we present AnICA, a tool taking inspiration from differential testing and abstract interpretation to systematically analyze inconsistencies among these code analyzers. Our evaluation shows that AnICA can summarize thousands of inconsistencies in a few dozen descriptions that directly lead to high-level insights into the different behavior of the tools. In several case studies, we further demonstrate how AnICA automatically finds and characterizes known and unknown bugs in llvm-mca, as well as a quirk in AMD's Zen microarchitectures.