NanoFlowNet: Real-time Dense Optical Flow on a Nano Quadcopter

TL;DR

NanoFlowNet is a lightweight neural network designed for real-time dense optical flow estimation on nano quadcopters, enabling autonomous navigation with low latency on ultra-low power hardware.

Contribution

The paper introduces NanoFlowNet, a novel lightweight CNN architecture optimized for edge devices, improving optical flow estimation for nano quadcopters without increasing latency.

Findings

High performance on MPI-Sintel dataset

Successful deployment on GAP8 microprocessor

Effective obstacle avoidance on Crazyflie quadcopter

Abstract

Nano quadcopters are small, agile, and cheap platforms that are well suited for deployment in narrow, cluttered environments. Due to their limited payload, these vehicles are highly constrained in processing power, rendering conventional vision-based methods for safe and autonomous navigation incompatible. Recent machine learning developments promise high-performance perception at low latency, while dedicated edge computing hardware has the potential to augment the processing capabilities of these limited devices. In this work, we present NanoFlowNet, a lightweight convolutional neural network for real-time dense optical flow estimation on edge computing hardware. We draw inspiration from recent advances in semantic segmentation for the design of this network. Additionally, we guide the learning of optical flow using motion boundary ground truth data, which improves performance with no impact on latency. Validation results on the MPI-Sintel dataset show the high performance of the proposed network given its constrained architecture. Additionally, we successfully demonstrate the capabilities of NanoFlowNet by deploying it on the ultra-low power GAP8 microprocessor and by applying it to vision-based obstacle avoidance on board a Bitcraze Crazyflie, a 34 g nano quadcopter.