Designing a Robust Low-Level Agnostic Controller for a Quadrotor with Actor-Critic Reinforcement Learning

TL;DR

This paper presents a domain-randomized Soft Actor-Critic based low-level controller for quadrotors, demonstrating improved robustness and adaptability in payload pickup/drop tasks with disturbances, outperforming traditional PID controllers.

Contribution

Introduces a domain randomization training method for a low-level RL controller that enhances robustness and generalization across varying quadrotor dynamics.

Findings

RL controller outperforms PID in payload tasks

Controller maintains performance across diverse quadrotor parameters

Domain randomization improves robustness to disturbances

Abstract

Purpose: Real-life applications using quadrotors introduce a number of disturbances and time-varying properties that pose a challenge to flight controllers. We observed that, when a quadrotor is tasked with picking up and dropping a payload, traditional PID and RL-based controllers found in literature struggle to maintain flight after the vehicle changes its dynamics due to interaction with this external object. Methods: In this work, we introduce domain randomization during the training phase of a low-level waypoint guidance controller based on Soft Actor-Critic. The resulting controller is evaluated on the proposed payload pick up and drop task with added disturbances that emulate real-life operation of the vehicle. Results & Conclusion: We show that, by introducing a certain degree of uncertainty in quadrotor dynamics during training, we can obtain a controller that is capable to perform the proposed task using a larger variation of quadrotor parameters. Additionally, the RL-based controller outperforms a traditional positional PID controller with optimized gains in this task, while remaining agnostic to different simulation parameters.