ProxFly: Robust Control for Close Proximity Quadcopter Flight via Residual Reinforcement Learning

TL;DR

ProxFly introduces a residual reinforcement learning controller that enhances close proximity quadcopter flight stability by compensating for disturbances without requiring communication or precise system models.

Contribution

It presents a novel residual RL-based control method that improves robustness and adaptability in close proximity quadcopter operations, including mid-air docking.

Findings

Effective in simulation across different proximities

Successfully stabilizes quadcopters in extreme proximity docking

Reduces unexplainable control signals and reliance on system identification

Abstract

This paper proposes the ProxFly, a residual deep Reinforcement Learning (RL)-based controller for close proximity quadcopter flight. Specifically, we design a residual module on top of a cascaded controller (denoted as basic controller) to generate high-level control commands, which compensate for external disturbances and thrust loss caused by downwash effects from other quadcopters. First, our method takes only the ego state and controllers' commands as inputs and does not rely on any communication between quadcopters, thereby reducing the bandwidth requirement. Through domain randomization, our method relaxes the requirement for accurate system identification and fine-tuned controller parameters, allowing it to adapt to changing system models. Meanwhile, our method not only reduces the proportion of unexplainable signals from the black box in control commands but also enables the RL training to skip the time-consuming exploration from scratch via guidance from the basic controller. We validate the effectiveness of the residual module in the simulation with different proximities. Moreover, we conduct the real close proximity flight test to compare ProxFly with the basic controller and an advanced model-based controller with complex aerodynamic compensation. Finally, we show that ProxFly can be used for challenging quadcopter mid-air docking, where two quadcopters fly in extreme proximity, and strong airflow significantly disrupts flight. However, our method can stabilize the quadcopter in this case and accomplish docking. The resources are available at https://github.com/ruiqizhang99/ProxFly.