Whole-Body Control Through Narrow Gaps From Pixels To Action

TL;DR

This paper presents a data-driven neural network approach for multirotor drones to navigate through narrow gaps using pixel inputs, combining reinforcement learning and trajectory optimization for effective control.

Contribution

It introduces a novel training pipeline that integrates model-free RL, online observation distillation, and model-based trajectory resets for complex narrow-gap flight skills.

Findings

Successful simulation of drone navigation through various narrow gaps

Effective distillation of policies from point clouds to pixel space

Enhanced training efficiency via trajectory resets

Abstract

Flying through body-size narrow gaps in the environment is one of the most challenging moments for an underactuated multirotor. We explore a purely data-driven method to master this flight skill in simulation, where a neural network directly maps pixels and proprioception to continuous low-level control commands. This learned policy enables whole-body control through gaps with different geometries demanding sharp attitude changes (e.g., near-vertical roll angle). The policy is achieved by successive model-free reinforcement learning (RL) and online observation space distillation. The RL policy receives (virtual) point clouds of the gaps' edges for scalable simulation and is then distilled into the high-dimensional pixel space. However, this flight skill is fundamentally expensive to learn by exploring due to restricted feasible solution space. We propose to reset the agent as states on the trajectories by a model-based trajectory optimizer to alleviate this problem. The presented training pipeline is compared with baseline methods, and ablation studies are conducted to identify the key ingredients of our method. The immediate next step is to scale up the variation of gap sizes and geometries in anticipation of emergent policies and demonstrate the sim-to-real transformation.