A Self-Driving Robot Using Deep Convolutional Neural Networks on Neuromorphic Hardware

TL;DR

This paper demonstrates a novel neuromorphic computing system that enables a self-driving robot to navigate steep mountain trails in real time using a deep convolutional neural network on IBM TrueNorth hardware.

Contribution

First implementation of a closed-loop, battery-powered neuromorphic system integrating IBM TrueNorth with an autonomous robot for real-time navigation.

Findings

Successfully navigated steep mountain trails in real time

First use of TrueNorth NS1e on a mobile platform in closed-loop control

Created a new dataset of mountain trail navigation behaviors

Abstract

Neuromorphic computing is a promising solution for reducing the size, weight and power of mobile embedded systems. In this paper, we introduce a realization of such a system by creating the first closed-loop battery-powered communication system between an IBM TrueNorth NS1e and an autonomous Android-Based Robotics platform. Using this system, we constructed a dataset of path following behavior by manually driving the Android-Based robot along steep mountain trails and recording video frames from the camera mounted on the robot along with the corresponding motor commands. We used this dataset to train a deep convolutional neural network implemented on the TrueNorth NS1e. The NS1e, which was mounted on the robot and powered by the robot's battery, resulted in a self-driving robot that could successfully traverse a steep mountain path in real time. To our knowledge, this represents the first time the TrueNorth NS1e neuromorphic chip has been embedded on a mobile platform under closed-loop control.