HARP: A Taxonomy for Heterogeneous and Hierarchical Processors for Mixed-reuse Workloads

TL;DR

This paper introduces HARP, a comprehensive taxonomy for classifying and analyzing hierarchical and heterogeneous accelerators designed for mixed-reuse AI workloads, addressing a relatively unexplored architectural space.

Contribution

HARP provides a systematic classification framework for hierarchical and heterogeneous processors, enabling better understanding and design of accelerators for diverse tensor workloads.

Findings

HARP captures various heterogeneity levels in accelerator architectures.

Modified Timeloop models effectively evaluate hierarchical and heterogeneous designs.

Study reveals impact of heterogeneity at different architecture levels.

Abstract

Artificial intelligence (AI) application domains consist of a mix of tensor operations with high and low arithmetic intensities (aka reuse). Hierarchical (i.e. compute along multiple levels of memory hierarchy) and heterogeneous (multiple different sub-accelerators) accelerators are emerging as a popular way to process mixed reuse workloads, and workloads which consist of tensor operators with diverse shapes. However, the space of hierarchical and/or heterogeneous processors (HHP's) is relatively under-explored. Prior works have proposed custom architectures to take advantage of heterogeneity to have multiple sub-accelerators that are efficient for different operator shapes. In this work, we propose HARP, a taxonomy to classify various hierarchical and heterogeneous accelerators and use the it to study the impact of heterogeneity at various levels in the architecture. HARP taxonomy captures various ways in which HHP's can be conceived, ranging from B100 cores with an "intra-node heterogeneity" between SM and tensor core to NeuPIM with cross-depth heterogeneity which occurs at different levels of memory hierarchy. We use Timeloop mapper to find the best mapping for sub-accelerators and also modify the Timeloop cost model to extend it to model hierarchical and heterogeneous accelerators.