MLIR-Forge: A Modular Framework for Language Smiths

TL;DR

MLIR-Forge is a flexible framework that simplifies the creation of random program generators for compiler IRs, enabling effective testing of compilers across multiple languages with minimal development effort.

Contribution

It introduces a modular, reusable approach to generate random programs for various IRs, reducing complexity and development time for testing compiler correctness.

Findings

Generated MLIR, WebAssembly, and DaCe programs with less than a week each.

Discovered 9 MLIR, 15 WebAssembly, and 774 DaCe bugs through differential testing.

Demonstrated effective bug detection across multiple IRs using the framework.

Abstract

Optimizing compilers are essential for the efficient and correct execution of software across various scientific fields. Domain-specific languages (DSL) typically use higher level intermediate representations (IR) in their compiler pipelines for domain-specific optimizations. As these IRs add to complexity, it is crucial to test them thoroughly. Random program generators have proven to be an effective tool to test compilers through differential and fuzz testing. However, developing specialized program generators for compiler IRs is not straightforward and demands considerable resources. We introduce MLIR-Forge, a novel random program generator framework that leverages the flexibility of MLIR, aiming to simplify the creation of specialized program generators. MLIR-Forge achieves this by splitting the generation process into fundamental building blocks that are language specific, and reusable program creation logic that constructs random programs from these building blocks. This hides complexity and furthermore, even the language specific components can be defined using a set of common tools. We demonstrate MLIR-Forge's capabilities by generating MLIR with built-in dialects, WebAssembly, and a data-centric program representation, DaCe -- requiring less than a week of development time in total for each of them. Using the generated programs we conduct differential testing and find 9 MLIR, 15 WebAssembly, and 774 DaCe groups of bugs with the corresponding program generators, after running them until the rate of new bugs stagnates.