Seeing Through Pixel Motion: Learning Obstacle Avoidance from Optical Flow with One Camera

TL;DR

This paper presents a novel end-to-end monocular optical flow-based obstacle avoidance system for quadrotors, enabling agile and robust flight in complex environments through a differentiable simulator and attention mechanisms.

Contribution

It introduces a new approach combining differentiable simulation, flow attention, and active sensing for monocular optical flow obstacle avoidance in drones.

Findings

Successful real-world deployment on FPV racing drone

Achieves agile flight at speeds up to 6m/s in cluttered environments

Demonstrates robustness and responsiveness in unknown environments

Abstract

Optical flow captures the motion of pixels in an image sequence over time, providing information about movement, depth, and environmental structure. Flying insects utilize this information to navigate and avoid obstacles, allowing them to execute highly agile maneuvers even in complex environments. Despite its potential, autonomous flying robots have yet to fully leverage this motion information to achieve comparable levels of agility and robustness. Challenges of control from optical flow include extracting accurate optical flow at high speeds, handling noisy estimation, and ensuring robust performance in complex environments. To address these challenges, we propose a novel end-to-end system for quadrotor obstacle avoidance using monocular optical flow. We develop an efficient differentiable simulator coupled with a simplified quadrotor model, allowing our policy to be trained directly through first-order gradient optimization. Additionally, we introduce a central flow attention mechanism and an action-guided active sensing strategy that enhances the policy's focus on task-relevant optical flow observations to enable more responsive decision-making during flight. Our system is validated both in simulation and the real world using an FPV racing drone. Despite being trained in a simple environment in simulation, our system is validated both in simulation and the real world using an FPV racing drone. Despite being trained in a simple environment in simulation, our system demonstrates agile and robust flight in various unknown, cluttered environments in the real world at speeds of up to 6m/s.