Atleus: Accelerating Transformers on the Edge Enabled by 3D Heterogeneous Manycore Architectures

TL;DR

Atleus is a novel 3D heterogeneous architecture designed to accelerate transformer models for edge applications, supporting both fine-tuning and inference with significant improvements in performance and energy efficiency.

Contribution

The paper introduces Atleus, a 3D heterogeneous platform optimized for transformer acceleration, incorporating non-volatile memory, systolic arrays, and a tailored NoC for edge deployment.

Findings

Achieves up to 56x performance improvement over existing solutions.

Attains 64.5x better energy efficiency compared to state-of-the-art.

Effectively supports model compression through quantization schemes.

Abstract

Transformer architectures have become the standard neural network model for various machine learning applications including natural language processing and computer vision. However, the compute and memory requirements introduced by transformer models make them challenging to adopt for edge applications. Furthermore, fine-tuning pre-trained transformers (e.g., foundation models) is a common task to enhance the model's predictive performance on specific tasks/applications. Existing transformer accelerators are oblivious to complexities introduced by fine-tuning. In this paper, we propose the design of a three-dimensional (3D) heterogeneous architecture referred to as Atleus that incorporates heterogeneous computing resources specifically optimized to accelerate transformer models for the dual purposes of fine-tuning and inference. Specifically, Atleus utilizes non-volatile memory and systolic array for accelerating transformer computational kernels using an integrated 3D platform. Moreover, we design a suitable NoC to achieve high performance and energy efficiency. Finally, Atleus adopts an effective quantization scheme to support model compression. Experimental results demonstrate that Atleus outperforms existing state-of-the-art by up to 56x and 64.5x in terms of performance and energy efficiency respectively