A 64mW DNN-based Visual Navigation Engine for Autonomous Nano-Drones

TL;DR

This paper presents the first demonstration of a DNN-based visual navigation system for autonomous nano-drones, achieving real-time performance within 64mW power constraints using a novel ultra-low-power platform.

Contribution

It introduces a complete methodology for executing complex deep neural networks onboard resource-constrained nano-drones, enabling autonomous navigation with minimal power consumption.

Findings

Achieves 6 fps real-time navigation with 64 mW power

Reaches 18 fps peak performance with low power use

Demonstrates successful autonomous flight using DNNs on nano-drones

Abstract

Fully-autonomous miniaturized robots (e.g., drones), with artificial intelligence (AI) based visual navigation capabilities are extremely challenging drivers of Internet-of-Things edge intelligence capabilities. Visual navigation based on AI approaches, such as deep neural networks (DNNs) are becoming pervasive for standard-size drones, but are considered out of reach for nanodrones with size of a few cm${}^\mathrm{2}$. In this work, we present the first (to the best of our knowledge) demonstration of a navigation engine for autonomous nano-drones capable of closed-loop end-to-end DNN-based visual navigation. To achieve this goal we developed a complete methodology for parallel execution of complex DNNs directly on-bard of resource-constrained milliwatt-scale nodes. Our system is based on GAP8, a novel parallel ultra-low-power computing platform, and a 27 g commercial, open-source CrazyFlie 2.0 nano-quadrotor. As part of our general methodology we discuss the software mapping techniques that enable the state-of-the-art deep convolutional neural network presented in [1] to be fully executed on-board within a strict 6 fps real-time constraint with no compromise in terms of flight results, while all processing is done with only 64 mW on average. Our navigation engine is flexible and can be used to span a wide performance range: at its peak performance corner it achieves 18 fps while still consuming on average just 3.5% of the power envelope of the deployed nano-aircraft.