A High-Level Compiler Integration Approach for Deep Learning Accelerators Supporting Abstraction and Optimization

TL;DR

This paper presents a high-level compiler integration method for deep learning accelerators that simplifies integration and automates optimization, achieving performance comparable to specialized manual tools.

Contribution

It introduces a TVM-based approach that abstracts integration complexities and incorporates automated tensor scheduling for GEMM-based accelerators.

Findings

Seamless integration of accelerators without deep compiler knowledge

Automated tensor scheduling with design space exploration

Performance comparable to manual toolchains on Gemmini

Abstract

The growing adoption of domain-specific architectures in edge computing platforms for deep learning has highlighted the efficiency of hardware accelerators. However, integrating custom accelerators into modern machine learning (ML) compilers remains a complex challenge due to the need for significant modifications in compilation layers and specialized scheduling techniques. Existing frameworks offer partial solutions and require users to navigate intricate compiler internals. In this paper, we introduce a TVM-based compilation integration approach that targets GEMM-based deep learning accelerators. Our approach abstracts the complexities of compiler integration, enabling seamless integration of accelerators without requiring in-depth knowledge of the underlying compiler. Furthermore, we extend and incorporate design space exploration tools, specifically CoSA, to automate efficient tensor scheduling, accounting for factors such as uneven mapping and double buffering. Our framework is benchmarked on the Gemmini accelerator, demonstrating performance comparable to its specialized manually implemented toolchain.