MinUn: Accurate ML Inference on Microcontrollers

TL;DR

MinUn is a novel TinyML framework that efficiently generates memory-frugal, high-precision inference code for microcontrollers, supporting emerging number representations and optimizing memory usage.

Contribution

It is the first TinyML framework to holistically address parametric number representations, tensor precision assignment, and memory fragmentation for microcontrollers.

Findings

Outperforms prior TinyML frameworks in efficiency

Supports emerging number representations like posits

Generates code for popular ARM microcontrollers

Abstract

Running machine learning inference on tiny devices, known as TinyML, is an emerging research area. This task requires generating inference code that uses memory frugally, a task that standard ML frameworks are ill-suited for. A deployment framework for TinyML must be a) parametric in the number representation to take advantage of the emerging representations like posits, b) carefully assign high-precision to a few tensors so that most tensors can be kept in low-precision while still maintaining model accuracy, and c) avoid memory fragmentation. We describe MinUn, the first TinyML framework that holistically addresses these issues to generate efficient code for ARM microcontrollers (e.g., Arduino Uno, Due and STM32H747) that outperforms the prior TinyML frameworks.