Control of a Quadrotor with Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning-based control method for quadrotors, featuring a new stable algorithm, demonstrating effective real-world stabilization and rapid computation suitable for real-time applications.

Contribution

The paper presents a novel reinforcement learning algorithm for quadrotor control that is more stable and applicable than existing methods, enabling real-time stabilization in complex scenarios.

Findings

The learned policy accurately responds to step inputs.

The policy stabilizes the quadrotor even under harsh initial conditions.

Evaluation time per step is only 7 microseconds, much faster than traditional methods.

Abstract

In this paper, we present a method to control a quadrotor with a neural network trained using reinforcement learning techniques. With reinforcement learning, a common network can be trained to directly map state to actuator command making any predefined control structure obsolete for training. Moreover, we present a new learning algorithm which differs from the existing ones in certain aspects. Our algorithm is conservative but stable for complicated tasks. We found that it is more applicable to controlling a quadrotor than existing algorithms. We demonstrate the performance of the trained policy both in simulation and with a real quadrotor. Experiments show that our policy network can react to step response relatively accurately. With the same policy, we also demonstrate that we can stabilize the quadrotor in the air even under very harsh initialization (manually throwing it upside-down in the air with an initial velocity of 5 m/s). Computation time of evaluating the policy is only 7 {\mu}s per time step which is two orders of magnitude less than common trajectory optimization algorithms with an approximated model.