Aggressive Perception-Aware Navigation using Deep Optical Flow Dynamics and PixelMPC

TL;DR

This paper introduces a real-time, vision-based control method combining deep optical flow dynamics with model predictive control, enabling high-speed drone racing while maintaining feature visibility for reliable localization.

Contribution

It proposes a novel, memory-efficient deep optical flow dynamics model integrated into an MPC framework, enhancing vision-based navigation in high-speed scenarios.

Findings

Effective real-time control in drone racing simulation

Improved visibility of key features during high-speed maneuvers

Enhanced reliability of vision-based localization methods

Abstract

Recently, vision-based control has gained traction by leveraging the power of machine learning. In this work, we couple a model predictive control (MPC) framework to a visual pipeline. We introduce deep optical flow (DOF) dynamics, which is a combination of optical flow and robot dynamics. Using the DOF dynamics, MPC explicitly incorporates the predicted movement of relevant pixels into the planned trajectory of a robot. Our implementation of DOF is memory-efficient, data-efficient, and computationally cheap so that it can be computed in real-time for use in an MPC framework. The suggested Pixel Model Predictive Control (PixelMPC) algorithm controls the robot to accomplish a high-speed racing task while maintaining visibility of the important features (gates). This improves the reliability of vision-based estimators for localization and can eventually lead to safe autonomous flight. The proposed algorithm is tested in a photorealistic simulation with a high-speed drone racing task.