High level programming abstractions for leveraging hierarchical memories with micro-core architectures

TL;DR

This paper explores high-level programming abstractions using Python to efficiently offload kernels to micro-core architectures with hierarchical memory, enabling larger data processing on resource-constrained devices.

Contribution

It introduces a pass-by-reference model and memory kind abstractions to improve programming ease and efficiency on micro-cores with complex memory hierarchies.

Findings

Able to process arbitrarily large datasets on micro-cores

Demonstrates performance and power efficiency of micro-core architectures

Shows potential for micro-cores to handle larger workloads with new abstractions

Abstract

Micro-core architectures combine many low memory, low power computing cores together in a single package. These are attractive for use as accelerators but due to limited on-chip memory and multiple levels of memory hierarchy, the way in which programmers offload kernels needs to be carefully considered. In this paper we use Python as a vehicle for exploring the semantics and abstractions of higher level programming languages to support the offloading of computational kernels to these devices. By moving to a pass by reference model, along with leveraging memory kinds, we demonstrate the ability to easily and efficiently take advantage of multiple levels in the memory hierarchy, even ones that are not directly accessible to the micro-cores. Using a machine learning benchmark, we perform experiments on both Epiphany-III and MicroBlaze based micro-cores, demonstrating the ability to compute with data sets of arbitrarily large size. To provide context of our results, we explore the performance and power efficiency of these technologies, demonstrating that whilst these two micro-core technologies are competitive within their own embedded class of hardware, there is still a way to go to reach HPC class GPUs.