Learning to Fly via Deep Model-Based Reinforcement Learning

TL;DR

This paper demonstrates that a quadrotor drone can learn to fly autonomously using deep model-based reinforcement learning with minimal training data, solely on onboard resources, without prior flight knowledge.

Contribution

It introduces a novel approach to drone control by learning a probabilistic model of dynamics directly from raw sensory data, enabling effective model-based RL for flight.

Findings

Achieved successful drone flight with less than 30 minutes of training.

Learned control policy using only onboard sensors and computation.

Demonstrated deployment on a self-built drone without prior knowledge.

Abstract

Learning to control robots without requiring engineered models has been a long-term goal, promising diverse and novel applications. Yet, reinforcement learning has only achieved limited impact on real-time robot control due to its high demand of real-world interactions. In this work, by leveraging a learnt probabilistic model of drone dynamics, we learn a thrust-attitude controller for a quadrotor through model-based reinforcement learning. No prior knowledge of the flight dynamics is assumed; instead, a sequential latent variable model, used generatively and as an online filter, is learnt from raw sensory input. The controller and value function are optimised entirely by propagating stochastic analytic gradients through generated latent trajectories. We show that "learning to fly" can be achieved with less than 30 minutes of experience with a single drone, and can be deployed solely using onboard computational resources and sensors, on a self-built drone.