Reinforcement Learning Position Control of a Quadrotor Using Soft Actor-Critic (SAC)

TL;DR

This paper introduces a reinforcement learning-based control architecture for quadrotors that directly manages the thrust vector using the Soft Actor-Critic algorithm, resulting in faster training and improved path-following accuracy.

Contribution

It presents a novel RL control approach that directly controls the thrust vector, integrating with PID for motor commands, and demonstrates superior training speed and path accuracy.

Findings

Faster training time compared to RPM controllers

Smoother and more accurate path-following in simulations

Effective control of thrust vector using SAC

Abstract

This paper proposes a new Reinforcement Learning (RL) based control architecture for quadrotors. With the literature focusing on controlling the four rotors' RPMs directly, this paper aims to control the quadrotor's thrust vector. The RL agent computes the percentage of overall thrust along the quadrotor's z-axis along with the desired Roll ($\phi$) and Pitch ($\theta$) angles. The agent then sends the calculated control signals along with the current quadrotor's Yaw angle ($\psi$) to an attitude PID controller. The PID controller then maps the control signals to motor RPMs. The Soft Actor-Critic algorithm, a model-free off-policy stochastic RL algorithm, was used to train the RL agents. Training results show the faster training time of the proposed thrust vector controller in comparison to the conventional RPM controllers. Simulation results show smoother and more accurate path-following for the proposed thrust vector controller.