Embedded Neural Networks for Robot Autonomy

TL;DR

This paper introduces an open-source library that embeds trained neural networks into robot materials, enabling real-time sensing and decision-making directly within the robot's structure using microcontrollers.

Contribution

The work presents a novel software tool for transferring trained neural networks onto microcontrollers embedded in robot materials, facilitating autonomous sensing and control.

Findings

Microcontrollers can match desktop accuracy in embedded neural network tasks.

Embedded neural networks enable real-time sensing within robot materials.

The library supports various neural network configurations and microcontroller types.

Abstract

We present a library to automatically embed signal processing and neural network predictions into the material robots are made of. Deep and shallow neural network models are first trained offline using state-of-the-art machine learning tools and then transferred onto general purpose microcontrollers that are co-located with a robot's sensors and actuators. We validate this approach using multiple examples: a smart robotic tire for terrain classification, a robotic finger sensor for load classification and a smart composite capable of regressing impact source localization. In each example, sensing and computation are embedded inside the material, creating artifacts that serve as stand-in replacement for otherwise inert conventional parts. The open source software library takes as inputs trained model files from higher level learning software, such as Tensorflow/Keras, and outputs code that is readable in a microcontroller that supports C. We compare the performance of this approach for various embedded platforms. In particular, we show that low-cost off-the-shelf microcontrollers can match the accuracy of a desktop computer, while being fast enough for real-time applications at different neural network configurations. We provide means to estimate the maximum number of parameters that the hardware will support based on the microcontroller's specifications.