Precise Aggressive Aerial Maneuvers with Sensorimotor Policies

TL;DR

This paper presents a reinforcement learning-based sensorimotor policy enabling quadrotors to perform precise, aggressive maneuvers through narrow gaps using onboard vision and proprioception, with successful real-world demonstrations.

Contribution

It introduces a novel RL training approach with model-based initialization for quadrotor gap traversal, achieving high accuracy without prior knowledge of gap position or orientation.

Findings

Quadrotors can navigate 5 cm wide gaps tilted at 90 degrees.

Policies generalize to moving gaps without additional training.

Method enables traversal of diverse narrow gaps with high repeatability.

Abstract

Precise aggressive maneuvers with lightweight onboard sensors remains a key bottleneck in fully exploiting the maneuverability of drones. Such maneuvers are critical for expanding the systems' accessible area by navigating through narrow openings in the environment. Among the most relevant problems, a representative one is aggressive traversal through narrow gaps with quadrotors under SE(3) constraints, which require the quadrotors to leverage a momentary tilted attitude and the asymmetry of the airframe to navigate through gaps. In this paper, we achieve such maneuvers by developing sensorimotor policies directly mapping onboard vision and proprioception into low-level control commands. The policies are trained using reinforcement learning (RL) with end-to-end policy distillation in simulation. We mitigate the fundamental hardness of model-free RL's exploration on the restricted solution space with an initialization strategy leveraging trajectories generated by a model-based planner. Careful sim-to-real design allows the policy to control a quadrotor through narrow gaps with low clearances and high repeatability. For instance, the proposed method enables a quadrotor to navigate a rectangular gap at a 5 cm clearance, tilted at up to 90-degree orientation, without knowledge of the gap's position or orientation. Without training on dynamic gaps, the policy can reactively servo the quadrotor to traverse through a moving gap. The proposed method is also validated by training and deploying policies on challenging tracks of narrow gaps placed closely. The flexibility of the policy learning method is demonstrated by developing policies for geometrically diverse gaps, without relying on manually defined traversal poses and visual features.