Dynamic Resource Partitioning for Multi-Tenant Systolic Array Based DNN Accelerator

TL;DR

This paper introduces a dynamic resource partitioning method for multi-tenant systolic array accelerators, significantly improving energy efficiency and computation speed when running multiple DNNs concurrently.

Contribution

It presents a novel dynamic partitioning algorithm and a modified dataflow for systolic array accelerators to enable efficient multi-DNN processing.

Findings

35% energy savings under heavy workloads

62% reduction in computation time under heavy workloads

Effective resource sharing for multiple DNNs

Abstract

Deep neural networks (DNN) have become significant applications in both cloud-server and edge devices. Meanwhile, the growing number of DNNs on those platforms raises the need to execute multiple DNNs on the same device. This paper proposes a dynamic partitioning algorithm to perform concurrent processing of multiple DNNs on a systolic-array-based accelerator. Sharing an accelerator's storage and processing resources across multiple DNNs increases resource utilization and reduces computation time and energy consumption. To this end, we propose a partitioned weight stationary dataflow with a minor modification in the logic of the processing element. We evaluate the energy consumption and computation time with both heavy and light workloads. Simulation results show a 35% and 62% improvement in energy consumption and 56% and 44% in computation time under heavy and light workloads, respectively, compared with single tenancy.