Toward Attention-based TinyML: A Heterogeneous Accelerated Architecture and Automated Deployment Flow

TL;DR

This paper presents a heterogeneous architecture combining RISC-V processors and accelerators for tinyML, enabling efficient deployment of attention-based models with high energy efficiency and throughput.

Contribution

It introduces an automated deployment flow and a heterogeneous architecture tailored for attention-based tinyML models, advancing the state-of-the-art in energy-efficient inference.

Findings

Achieved 2960 GOp/J energy efficiency

Reached 154 GOp/s throughput

Demonstrated end-to-end 8-bit Transformer inference

Abstract

One of the challenges for Tiny Machine Learning (tinyML) is keeping up with the evolution of Machine Learning models from Convolutional Neural Networks to Transformers. We address this by leveraging a heterogeneous architectural template coupling RISC-V processors with hardwired accelerators supported by an automated deployment flow. We demonstrate Attention-based models in a tinyML power envelope with an octa-core cluster coupled with an accelerator for quantized Attention. Our deployment flow enables end-to-end 8-bit Transformer inference, achieving leading-edge energy efficiency and throughput of 2960 GOp/J and 154 GOp/s (0.65 V, 22 nm FD-SOI technology).