All Eyes, no IMU: Learning Flight Attitude from Vision Alone

TL;DR

This paper introduces a novel vision-only flight control system for quadrotors using an event camera and neural networks, eliminating the need for inertial sensors and enabling autonomous insect-scale flying robots.

Contribution

It presents the first vision-based approach for flight control in drones, demonstrating real-world viability and analyzing generalization across environments.

Findings

Event camera and neural network can replace inertial sensors for attitude estimation.

Networks with memory and horizon cues perform best, but narrower FOV networks generalize better.

Real-world tests confirm effective autonomous flight without inertial measurement units.

Abstract

Vision is an essential part of attitude control for many flying animals, some of which have no dedicated sense of gravity. Flying robots, on the other hand, typically depend heavily on accelerometers and gyroscopes for attitude stabilization. In this work, we present the first vision-only approach to flight control for use in generic environments. We show that a quadrotor drone equipped with a downward-facing event camera can estimate its attitude and rotation rate from just the event stream, enabling flight control without inertial sensors. Our approach uses a small recurrent convolutional neural network trained through supervised learning. Real-world flight tests demonstrate that our combination of event camera and low-latency neural network is capable of replacing the inertial measurement unit in a traditional flight control loop. Furthermore, we investigate the network's generalization across different environments, and the impact of memory and different fields of view. While networks with memory and access to horizon-like visual cues achieve best performance, variants with a narrower field of view achieve better relative generalization. Our work showcases vision-only flight control as a promising candidate for enabling autonomous, insect-scale flying robots.