A VM/Containerized Approach for Scaling TinyML Applications

TL;DR

This paper introduces a containerization approach for TinyML applications, enabling scalable deployment, updating, and monitoring of machine learning models across diverse edge devices through open-source tools and a novel container format called Runes.

Contribution

It presents a new container-based framework for deploying TinyML models, facilitating cross-device compatibility and management in IoT environments.

Findings

Enables deployment of ML models via containers on edge devices

Supports cross-platform compatibility in IoT ecosystems

Provides open-source tools for model deployment and monitoring

Abstract

Although deep neural networks are typically computationally expensive to use, technological advances in both the design of hardware platforms and of neural network architectures, have made it possible to use powerful models on edge devices. To enable widespread adoption of edge based machine learning, we introduce a set of open-source tools that make it easy to deploy, update and monitor machine learning models on a wide variety of edge devices. Our tools bring the concept of containerization to the TinyML world. We propose to package ML and application logic as containers called Runes to deploy onto edge devices. The containerization allows us to target a fragmented Internet-of-Things (IoT) ecosystem by providing a common platform for Runes to run across devices.