A System Level Compiler for Massively-Parallel, Spatial, Dataflow Architectures

TL;DR

This paper introduces MACH, a novel system-level compiler designed to efficiently translate high-level code into machine-specific instructions for massively-parallel, spatial dataflow architectures like the Wafer Scale Engine, while also supporting traditional devices.

Contribution

MACH is a flexible compiler that bridges high-level languages and hardware-specific code for diverse architectures using a Virtual Machine and domain-specific language.

Findings

Successfully lowered NumPy dense tensor code to Cerebras hardware languages.

Supports multiple architectures and data mappings.

Operates on both spatial dataflow and traditional devices.

Abstract

We have developed a novel compiler called the Multiple-Architecture Compiler for Advanced Computing Hardware (MACH) designed specifically for massively-parallel, spatial, dataflow architectures like the Wafer Scale Engine. Additionally, MACH can execute code on traditional unified-memory devices. MACH addresses the complexities in compiling for spatial architectures through a conceptual Virtual Machine, a flexible domain-specific language, and a compiler that can lower high-level languages to machine-specific code in compliance with the Virtual Machine concept. While MACH is designed to be operable on several architectures and provide the flexibility for several standard and user-defined data mappings, we introduce the concept with dense tensor examples from NumPy and show lowering to the Wafer Scale Engine by targeting Cerebras' hardware specific languages.