How to Manage Tiny Machine Learning at Scale: An Industrial Perspective

TL;DR

This paper introduces a semantic web-based framework for managing TinyML models and IoT devices at scale, addressing heterogeneity and resource constraints in industrial deployments.

Contribution

It proposes an ontology and knowledge graph approach to facilitate discovery, benchmarking, and reuse of TinyML components across diverse IoT devices.

Findings

Semantic schema for neural networks aligned with W3C Thing Description

Knowledge graph of 23 ML models and 6 IoT devices

Demonstrated management framework through three case studies

Abstract

Tiny machine learning (TinyML) has gained widespread popularity where machine learning (ML) is democratized on ubiquitous microcontrollers, processing sensor data everywhere in real-time. To manage TinyML in the industry, where mass deployment happens, we consider the hardware and software constraints, ranging from available onboard sensors and memory size to ML-model architectures and runtime platforms. However, Internet of Things (IoT) devices are typically tailored to specific tasks and are subject to heterogeneity and limited resources. Moreover, TinyML models have been developed with different structures and are often distributed without a clear understanding of their working principles, leading to a fragmented ecosystem. Considering these challenges, we propose a framework using Semantic Web technologies to enable the joint management of TinyML models and IoT devices at scale, from modeling information to discovering possible combinations and benchmarking, and eventually facilitate TinyML component exchange and reuse. We present an ontology (semantic schema) for neural network models aligned with the World Wide Web Consortium (W3C) Thing Description, which semantically describes IoT devices. Furthermore, a Knowledge Graph of 23 publicly available ML models and six IoT devices were used to demonstrate our concept in three case studies, and we shared the code and examples to enhance reproducibility: https://github.com/Haoyu-R/How-to-Manage-TinyML-at-Scale