Compiler Toolchains for Deep Learning Workloads on Embedded Platforms

TL;DR

This paper surveys and benchmarks open-source deep learning compiler toolchains for embedded platforms, and demonstrates a compilation flow for heterogeneous devices to aid hardware developers.

Contribution

It provides a comprehensive benchmark of existing tools and offers a practical implementation guide for compiling deep learning models on heterogeneous embedded hardware.

Findings

Benchmark results highlight performance differences among toolchains.

Demonstrated compilation flow facilitates hardware-specific deep learning deployment.

Insights into capabilities of open-source compiler solutions for embedded AI.

Abstract

As the usage of deep learning becomes increasingly popular in mobile and embedded solutions, it is necessary to convert the framework-specific network representations into executable code for these embedded platforms. This paper consists of two parts: The first section is made up of a survey and benchmark of the available open source deep learning compiler toolchains, which focus on the capabilities and performance of the individual solutions in regard to targeting embedded devices and microcontrollers that are combined with a dedicated accelerator in a heterogeneous fashion. The second part explores the implementation and evaluation of a compilation flow for such a heterogeneous device and reuses one of the existing toolchains to demonstrate the necessary steps for hardware developers that plan to build a software flow for their own hardware.