CODA: Enabling Co-location of Computation and Data for Near-Data Processing

TL;DR

This paper introduces techniques to co-locate computation and data in memory modules, enhancing near-data processing efficiency by optimizing data placement and access patterns, leading to significant performance improvements.

Contribution

It proposes a novel approach to selectively localize or distribute memory pages based on access patterns, enabling effective co-location of computation and data in multi-memory systems.

Findings

Performance improved by 31%

Remote data accesses reduced by 38%

Effective data placement enhances near-data processing

Abstract

Recent studies have demonstrated that near-data processing (NDP) is an effective technique for improving performance and energy efficiency of data-intensive workloads. However, leveraging NDP in realistic systems with multiple memory modules introduces a new challenge. In today's systems, where no computation occurs in memory modules, the physical address space is interleaved at a fine granularity among all memory modules to help improve the utilization of processor-memory interfaces by distributing the memory traffic. However, this is at odds with efficient use of NDP, which requires careful placement of data in memory modules such that near-data computations and their exclusively used data can be localized in individual memory modules, while distributing shared data among memory modules to reduce hotspots. In order to address this new challenge, we propose a set of techniques that (1) enable collections of OS pages to either be fine-grain interleaved among memory modules (as is done today) or to be placed contiguously on individual memory modules (as is desirable for NDP private data), and (2) decide whether to localize or distribute each memory object based on its anticipated access pattern and steer computations to the memory where the data they access is located. Our evaluations across a wide range of workloads show that the proposed mechanism improves performance by 31% and reduces 38% remote data accesses over a baseline system that cannot exploit computate-data affinity characteristics.