Newton-PnP: Real-time Visual Navigation for Autonomous Toy-Drones

TL;DR

This paper introduces Newton-PnP, a real-time, provably correct pose estimation algorithm designed for low-power IoT devices, enabling autonomous indoor navigation for lightweight toy drones using onboard processing.

Contribution

The paper presents a novel real-time PnP solver with theoretical guarantees optimized for weak IoT devices, facilitating autonomous drone navigation without external sensors.

Findings

Runs in real-time on low-power IoT hardware

Provides provable guarantees for correctness and efficiency

Successfully enables indoor navigation for toy drones

Abstract

The Perspective-n-Point problem aims to estimate the relative pose between a calibrated monocular camera and a known 3D model, by aligning pairs of 2D captured image points to their corresponding 3D points in the model. We suggest an algorithm that runs on weak IoT devices in real-time but still provides provable theoretical guarantees for both running time and correctness. Existing solvers provide only one of these requirements. Our main motivation was to turn the popular DJI's Tello Drone (<90gr, <\$100) into an autonomous drone that navigates in an indoor environment with no external human/laptop/sensor, by simply attaching a Raspberry PI Zero (<9gr, <\$25) to it. This tiny micro-processor takes as input a real-time video from a tiny RGB camera, and runs our PnP solver on-board. Extensive experimental results, open source code, and a demonstration video are included.