Compilation and Optimizations for Efficient Machine Learning on Embedded Systems

TL;DR

This paper presents a comprehensive set of design methodologies, including model optimization and hardware acceleration, to enable efficient deployment of deep neural networks on resource-constrained embedded systems.

Contribution

It introduces novel model and hardware design strategies specifically tailored for embedded systems to improve efficiency and performance of machine learning applications.

Findings

Enhanced ML models with reduced computational complexity

Customized hardware accelerators for embedded platforms

Improved inference efficiency and resource utilization

Abstract

Deep Neural Networks (DNNs) have achieved great success in a variety of machine learning (ML) applications, delivering high-quality inferencing solutions in computer vision, natural language processing, and virtual reality, etc. However, DNN-based ML applications also bring much increased computational and storage requirements, which are particularly challenging for embedded systems with limited compute/storage resources, tight power budgets, and small form factors. Challenges also come from the diverse application-specific requirements, including real-time responses, high-throughput performance, and reliable inference accuracy. To address these challenges, we introduce a series of effective design methodologies, including efficient ML model designs, customized hardware accelerator designs, and hardware/software co-design strategies to enable efficient ML applications on embedded systems.