EdgeMark: An Automation and Benchmarking System for Embedded Artificial Intelligence Tools

TL;DR

This paper reviews existing embedded AI tools, introduces EdgeMark for automating deployment and benchmarking of ML models on embedded devices, and provides experimental results comparing popular tools to guide selection.

Contribution

It presents EdgeMark, an open-source system that automates and benchmarks embedded AI tools, enhancing reproducibility and scalability in TinyML workflows.

Findings

EdgeMark simplifies deployment workflows.

Benchmark results compare TFLM, Edge Impulse, Ekkono, and Renesas eAI.

Insights into tool strengths and weaknesses.

Abstract

The integration of artificial intelligence (AI) into embedded devices, a paradigm known as embedded artificial intelligence (eAI) or tiny machine learning (TinyML), is transforming industries by enabling intelligent data processing at the edge. However, the many tools available in this domain leave researchers and developers wondering which one is best suited to their needs. This paper provides a review of existing eAI tools, highlighting their features, trade-offs, and limitations. Additionally, we introduce EdgeMark, an open-source automation system designed to streamline the workflow for deploying and benchmarking machine learning (ML) models on embedded platforms. EdgeMark simplifies model generation, optimization, conversion, and deployment while promoting modularity, reproducibility, and scalability. Experimental benchmarking results showcase the performance of widely used eAI tools, including TensorFlow Lite Micro (TFLM), Edge Impulse, Ekkono, and Renesas eAI Translator, across a wide range of models, revealing insights into their relative strengths and weaknesses. The findings provide guidance for researchers and developers in selecting the most suitable tools for specific application requirements, while EdgeMark lowers the barriers to adoption of eAI technologies.