TATAA: Programmable Mixed-Precision Transformer Acceleration with a Transformable Arithmetic Architecture
Jiajun Wu, Mo Song, Jingmin Zhao, Yizhao Gao, Jia Li, Hayden, Kwok-Hay So

TL;DR
TATAA is a programmable accelerator that efficiently combines 8-bit integer and bfloat16 floating-point arithmetic to accelerate transformer models, maintaining accuracy while improving throughput and power efficiency.
Contribution
The paper introduces TATAA, a transformable hardware architecture supporting mixed-precision computations for transformers, with an end-to-end compiler for flexible model mapping.
Findings
Achieves up to 1.45x higher throughput than related work.
Maintains less than 1.16% accuracy drop across various applications.
Outperforms modern GPUs in power efficiency by 2.19x.
Abstract
Modern transformer-based deep neural networks present unique technical challenges for effective acceleration in real-world applications. Apart from the vast amount of linear operations needed due to their sizes, modern transformer models are increasingly reliance on precise non-linear computations that make traditional low-bitwidth quantization methods and fixed-dataflow matrix accelerators ineffective for end-to-end acceleration. To address this need to accelerate both linear and non-linear operations in a unified and programmable framework, this paper introduces TATAA. TATAA employs 8-bit integer (int8) arithmetic for quantized linear layer operations through post-training quantization, while it relies on bfloat16 floating-point arithmetic to approximate non-linear layers of a transformer model. TATAA hardware features a transformable arithmetic architecture that supports both formats…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAnalog and Mixed-Signal Circuit Design · Advancements in PLL and VCO Technologies · Advanced Electrical Measurement Techniques
