Taming Offload Overheads in a Massively Parallel Open-Source RISC-V MPSoC: Analysis and Optimization

TL;DR

This paper analyzes and optimizes offload overheads in a large-scale RISC-V heterogeneous MPSoC, demonstrating hardware-software co-design and multicast enhancements that significantly improve application performance.

Contribution

It provides a detailed cycle-accurate analysis of offload overheads and introduces hardware and software co-design strategies, including multicast Network-on-Chip, to reduce overheads and improve performance.

Findings

Multicast Network-on-Chip reduces offload overheads by up to 2.3x.

The proposed model estimates application runtime with less than 15% error.

Hardware-software co-design effectively mitigates synchronization and communication costs.

Abstract

Heterogeneous multi-core architectures combine on a single chip a few large, general-purpose host cores, optimized for single-thread performance, with (many) clusters of small, specialized, energy-efficient accelerator cores for data-parallel processing. Offloading a computation to the many-core acceleration fabric implies synchronization and communication overheads which can hamper overall performance and efficiency, particularly for small and fine-grained parallel tasks. In this work, we present a detailed, cycle-accurate quantitative analysis of the offload overheads on Occamy, an open-source massively parallel RISC-V based heterogeneous MPSoC. We study how the overheads scale with the number of accelerator cores. We explore an approach to drastically reduce these overheads by co-designing the hardware and the offload routines. Notably, we demonstrate that by incorporating multicast capabilities into the Network-on-Chip of a large (200+ cores) accelerator fabric we can improve offloaded application runtimes by as much as 2.3x, restoring more than 70% of the ideally attainable speedups. Finally, we propose a quantitative model to estimate the runtime of selected applications accounting for the offload overheads, with an error consistently below 15%.