VUSA: Virtually Upscaled Systolic Array Architecture to Exploit Unstructured Sparsity in AI Acceleration

TL;DR

VUSA is a novel systolic array architecture that dynamically adapts to unstructured sparsity in neural networks, significantly improving area and power efficiency while maintaining peak performance for diverse DNN workloads.

Contribution

It introduces a virtually upscaled systolic array design that exploits unstructured sparsity to enhance efficiency without sacrificing general applicability.

Findings

Achieves 37% area savings and 68% power savings at the same peak performance.

Supports acceleration for any DNN, regardless of sparsity level.

Demonstrates viability in a 16-nm technology process.

Abstract

Leveraging high degrees of unstructured sparsity is a promising approach to enhance the efficiency of deep neural network DNN accelerators - particularly important for emerging Edge-AI applications. We introduce VUSA, a systolic-array architecture that virtually grows based on the present sparsity to perform larger matrix multiplications with the same number of physical multiply-accumulate MAC units. The proposed architecture achieves saving by 37% and 68% in area and power efficiency, respectively, at the same peak-performance, compared to a baseline systolic array architecture in a commercial 16-nm technology. Still, the proposed architecture supports acceleration for any DNN with any sparsity - even no sparsity at all. Thus, the proposed architecture is application-independent, making it viable for general-purpose AI acceleration.