Time-attenuating Twin Delayed DDPG Reinforcement Learning for Trajectory Tracking Control of Quadrotors

TL;DR

This paper introduces a novel time-attenuating twin delayed DDPG algorithm for quadrotor trajectory tracking, demonstrating improved accuracy, efficiency, and robustness against environmental noise through simulation tests.

Contribution

It proposes a model-free reinforcement learning method with a time decay strategy to enhance quadrotor trajectory tracking performance.

Findings

Significantly reduced tracking error

Operation time is one-tenth of traditional algorithms

Improved training efficiency and convergence stability

Abstract

Continuous trajectory tracking control of quadrotors is complicated when considering noise from the environment. Due to the difficulty in modeling the environmental dynamics, tracking methodologies based on conventional control theory, such as model predictive control, have limitations on tracking accuracy and response time. We propose a Time-attenuating Twin Delayed DDPG, a model-free algorithm that is robust to noise, to better handle the trajectory tracking task. A deep reinforcement learning framework is constructed, where a time decay strategy is designed to avoid trapping into local optima. The experimental results show that the tracking error is significantly small, and the operation time is one-tenth of that of a traditional algorithm. The OpenAI Mujoco tool is used to verify the proposed algorithm, and the simulation results show that, the proposed method can significantly improve the training efficiency and effectively improve the accuracy and convergence stability.