Deep Learning on Home Drone: Searching for the Optimal Architecture

TL;DR

This paper presents a real-time semantic segmentation system for a micro-computer attached to a toy drone, enabling autonomous object detection and classification without external sensors, suitable for IoT applications.

Contribution

It introduces the first real-time deep learning system on a micro-computer for a toy drone, optimizing network architecture for speed and accuracy tradeoffs.

Findings

Achieved real-time semantic segmentation on Raspberry Pi Zero v2

Demonstrated autonomous drone object detection in practical scenarios

Developed an efficient search algorithm for optimal network architecture

Abstract

We suggest the first system that runs real-time semantic segmentation via deep learning on a weak micro-computer such as the Raspberry Pi Zero v2 (whose price was \$15) attached to a toy-drone. In particular, since the Raspberry Pi weighs less than $16$ grams, and its size is half of a credit card, we could easily attach it to the common commercial DJI Tello toy-drone (<\$100, <90 grams, 98 $\times$ 92.5 $\times$ 41 mm). The result is an autonomous drone (no laptop nor human in the loop) that can detect and classify objects in real-time from a video stream of an on-board monocular RGB camera (no GPS or LIDAR sensors). The companion videos demonstrate how this Tello drone scans the lab for people (e.g. for the use of firefighters or security forces) and for an empty parking slot outside the lab. Existing deep learning solutions are either much too slow for real-time computation on such IoT devices, or provide results of impractical quality. Our main challenge was to design a system that takes the best of all worlds among numerous combinations of networks, deep learning platforms/frameworks, compression techniques, and compression ratios. To this end, we provide an efficient searching algorithm that aims to find the optimal combination which results in the best tradeoff between the network running time and its accuracy/performance.