A Benchmark Comparison of Learned Control Policies for Agile Quadrotor Flight

TL;DR

This paper benchmarks various learned control policies for agile quadrotor flight, demonstrating that policies commanding body-rates and thrust transfer more robustly from simulation to real-world at high speeds.

Contribution

It provides the first comprehensive benchmark comparison of learned control policies for quadrotors and shows the effectiveness of body-rate and thrust control policies in real-world high-speed flight.

Findings

Training body-rate and thrust policies improves sim-to-real transfer.

Deep reinforcement learning enables real-world control at speeds over 45 km/h.

Benchmark results highlight the robustness of specific control policy architectures.

Abstract

Quadrotors are highly nonlinear dynamical systems that require carefully tuned controllers to be pushed to their physical limits. Recently, learning-based control policies have been proposed for quadrotors, as they would potentially allow learning direct mappings from high-dimensional raw sensory observations to actions. Due to sample inefficiency, training such learned controllers on the real platform is impractical or even impossible. Training in simulation is attractive but requires to transfer policies between domains, which demands trained policies to be robust to such domain gap. In this work, we make two contributions: (i) we perform the first benchmark comparison of existing learned control policies for agile quadrotor flight and show that training a control policy that commands body-rates and thrust results in more robust sim-to-real transfer compared to a policy that directly specifies individual rotor thrusts, (ii) we demonstrate for the first time that such a control policy trained via deep reinforcement learning can control a quadrotor in real-world experiments at speeds over 45km/h.