TensorLib: A Spatial Accelerator Generation Framework for Tensor Algebra

TL;DR

TensorLib is a framework that automates the generation of spatial hardware accelerators for tensor algebra, enabling efficient design space exploration and achieving significant performance improvements on FPGA.

Contribution

It introduces a systematic framework using Space-Time Transformation and parameterized modules to automate hardware generation for tensor algebra accelerators.

Findings

Achieves 21% performance improvement on FPGA

Automates hardware design with dataflow exploration

Reduces development time for spatial accelerators

Abstract

Tensor algebra finds applications in various domains, and these applications, especially when accelerated on spatial hardware accelerators, can deliver high performance and low power. Spatial hardware accelerator exhibits complex design space. Prior approaches based on manual implementation lead to low programming productivity, rendering thorough design space exploration impossible. In this paper, we propose TensorLib, a framework for generating spatial hardware accelerator for tensor algebra applications. TensorLib is motivated by the observation that, different dataflows share common hardware modules, which can be reused across different designs. To build such a framework, TensorLib first uses Space-Time Transformation to explore different dataflows, which can compactly represent the hardware dataflow using a simple transformation matrix. Next, we identify the common structures of different dataflows and build parameterized hardware module templates with Chisel. Our generation framework can select the needed hardware modules for each dataflow, connect the modules using a specified interconnection pattern, and automatically generate the complete hardware accelerator design. TensorLib remarkably improves the productivity for the development and optimization of spatial hardware architecture, providing a rich design space with trade-offs in performance, area, and power. Experiments show that TensorLib can automatically generate hardware designs with different dataflows and achieve 21\% performance improvement on FPGA compared to the state-of-the-arts.