On Memory Codelets: Prefetching, Recoding, Moving and Streaming Data

TL;DR

This paper introduces Memory Codelets within the Codelet PXM to enhance memory utilization through prefetching, recoding, moving, and streaming data, addressing limitations of traditional strategies in heterogeneous and complex memory systems.

Contribution

It extends the Codelet Model to include Memory Codelets, enabling well-defined memory operations for improved data management in diverse computing environments.

Findings

Memory Codelets improve data movement efficiency.

The approach enhances memory utilization in heterogeneous systems.

It provides a structured way to optimize memory access patterns.

Abstract

For decades, memory capabilities have scaled up much slower than compute capabilities, leaving memory utilization as a major bottleneck. Prefetching and cache hierarchies mitigate this in applications with easily predictable memory accesses or those with high locality. In other applications like sparse linear algebra or graph-based applications, these strategies do not achieve effective utilization of memory. This is the case for the von Neumann model of computation, but other program execution models (PXM) provide different opportunities. Furthermore, the problem is complicated by increasing levels of heterogeneity and devices' varying memory subsystems. The Codelet PXM presented in this paper provides a program structure that allows for well-defined prefetching, streaming, and recoding operations to improve memory utilization and efficiently coordinate data movement with respect to computation. We propose the Memory Codelet, an extension to the original Codelet Model, to provide users these functionalities in a well-defined manner within the Codelet PXM.