TinyD\'ej\`aVu: Smaller Memory Footprint & Faster Inference on Sensor Data Streams with Always-On Microcontrollers

TL;DR

TinyDéjàVu is a framework that significantly reduces RAM usage and redundant computation for tiny neural networks on microcontrollers, enabling more efficient always-on sensor data inference.

Contribution

It introduces novel algorithms and an open-source framework that drastically cut memory footprint and computation for tiny ML models on resource-constrained microcontrollers.

Findings

Reduces RAM usage by over 60%.

Eliminates up to 90% of redundant computation.

Provides reproducible benchmarks on hardware.

Abstract

Always-on sensors are increasingly expected to embark a variety of tiny neural networks and to continuously perform inference on time-series of the data they sense. In order to fit lifetime and energy consumption requirements when operating on battery, such hardware uses microcontrollers (MCUs) with tiny memory budget e.g., 128kB of RAM. In this context, optimizing data flows across neural network layers becomes crucial. In this paper, we introduce TinyD\'ej\`aVu, a new framework and novel algorithms we designed to drastically reduce the RAM footprint required by inference using various tiny ML models for sensor data time-series on typical microcontroller hardware. We publish the implementation of TinyD\'ej\`aVu as open source, and we perform reproducible benchmarks on hardware. We show that TinyD\'ej\`aVu can save more than 60% of RAM usage and eliminate up to 90% of redundant compute on overlapping sliding window inputs.