CoMoNM: A Cost Modeling Framework for Compute-Near-Memory Systems

TL;DR

CoMoNM is a fast, generic cost modeling framework that estimates execution time for compute-near-memory systems, aiding optimization and design decisions with high accuracy and significantly reduced simulation time.

Contribution

It introduces CoMoNM, a novel, hardware-agnostic cost model for CNM systems that enables rapid execution time estimation to improve compiler offloading decisions.

Findings

Estimation errors within 7.80% and 2.99% compared to real systems.

Provides execution time estimates seven orders of magnitude faster than detailed simulators.

Seamlessly integrates into CNM compilers for improved optimization.

Abstract

Compute-Near-Memory (CNM) systems offer a promising approach to mitigate the von Neumann bottleneck by bringing computational units closer to data. However, optimizing for these architectures remains challenging due to their unique hardware and programming models. Existing CNM compilers often rely on manual programmer annotations for offloading and optimizations. Automating these decisions by exploring the optimization space, common in CPU/GPU systems, is difficult for CNMs as constructing and navigating the transformation space is tedious and time consuming. This is particularly the case during system-level design, where evaluation requires time-consuming simulations. To address this, we present CoMoNM, a generic cost modeling framework for CNM systems for execution time estimation in milliseconds. It takes a high-level, hardware-agnostic application representation, target system specifications, and a mapping specification as input and estimates the execution time for the given application on the target CNM system. We show how CoMoNM can be seamlessly integrated into state-of-the-art CNM compilers, providing improved offloading decisions. Evaluation on established benchmarks for CNM shows estimation errors within 7.80% and 2.99%, when compared to the real UPMEM CNM system and Samsung's HBM-PIM simulator. Notably, CoMoNM delivers estimates seven orders of magnitude faster compared to the UPMEM and HBM-PIM simulators.