Reactive Aerobatic Flight via Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning framework for quadrotors to perform complex aerobatic maneuvers end-to-end, achieving high agility and robustness through curriculum learning and sim-to-real transfer, validated by real-world experiments.

Contribution

It presents a novel RL-based approach with automated curriculum learning for end-to-end aerobatic flight, surpassing traditional modular methods in agility and robustness.

Findings

First autonomous inverted flight with reactive navigation

Successful sim-to-real transfer using domain randomization

Demonstrated high agility in real-world experiments

Abstract

Quadrotors have demonstrated remarkable versatility, yet their full aerobatic potential remains largely untapped due to inherent underactuation and the complexity of aggressive maneuvers. Traditional approaches, separating trajectory optimization and tracking control, suffer from tracking inaccuracies, computational latency, and sensitivity to initial conditions, limiting their effectiveness in dynamic, high-agility scenarios. Inspired by recent breakthroughs in data-driven methods, we propose a reinforcement learning-based framework that directly maps drone states and aerobatic intentions to control commands, eliminating modular separation to enable quadrotors to perform end-to-end policy optimization for extreme aerobatic maneuvers. To ensure efficient and stable training, we introduce an automated curriculum learning strategy that dynamically adjusts aerobatic task difficulty. Enabled by domain randomization for robust zero-shot sim-to-real transfer, our approach is validated in demanding real-world experiments, including the first demonstration of a drone autonomously performing continuous inverted flight while reactively navigating a moving gate, showcasing unprecedented agility.