Shisha: Online scheduling of CNN pipelines on heterogeneous architectures

TL;DR

Shisha is an online scheduling method for CNN pipelines on heterogeneous chiplet architectures, optimizing workload distribution quickly and effectively, significantly outperforming traditional exploration algorithms in convergence time.

Contribution

Shisha introduces a fast online scheduling approach for CNNs on chiplet architectures, improving exploration speed and solution quality over existing heuristics.

Findings

Shisha reduces exploration time by approximately 35 times.

Shisha produces better scheduling solutions than other heuristic algorithms.

The approach effectively handles heterogeneity in compute and memory bandwidth.

Abstract

Chiplets have become a common methodology in modern chip design. Chiplets improve yield and enable heterogeneity at the level of cores, memory subsystem and the interconnect. Convolutional Neural Networks (CNNs) have high computational, bandwidth and memory capacity requirements owing to the increasingly large amount of weights. Thus to exploit chiplet-based architectures, CNNs must be optimized in terms of scheduling and workload distribution among computing resources. We propose Shisha, an online approach to generate and schedule parallel CNN pipelines on chiplet architectures. Shisha targets heterogeneity in compute performance and memory bandwidth and tunes the pipeline schedule through a fast online exploration technique. We compare Shisha with Simulated Annealing, Hill Climbing and Pipe-Search. On average, the convergence time is improved by ~35x in Shisha compared to other exploration algorithms. Despite the quick exploration, Shisha's solution is often better than that of other heuristic exploration algorithms.