Control of a Twin Rotor using Twin Delayed Deep Deterministic Policy Gradient (TD3)

TL;DR

This paper introduces a reinforcement learning approach using TD3 to control and stabilize the complex, nonlinear Twin Rotor Aerodynamic System, demonstrating its effectiveness through simulations and real-world experiments against traditional controllers.

Contribution

It applies the TD3 reinforcement learning algorithm to control the Twin Rotor System, showcasing its advantages over conventional methods in complex, nonlinear environments.

Findings

RL control outperforms PID in stability and accuracy

Effective disturbance rejection demonstrated in simulations

Successful real-world implementation on laboratory setup

Abstract

This paper proposes a reinforcement learning (RL) framework for controlling and stabilizing the Twin Rotor Aerodynamic System (TRAS) at specific pitch and azimuth angles and tracking a given trajectory. The complex dynamics and non-linear characteristics of the TRAS make it challenging to control using traditional control algorithms. However, recent developments in RL have attracted interest due to their potential applications in the control of multirotors. The Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm was used in this paper to train the RL agent. This algorithm is used for environments with continuous state and action spaces, similar to the TRAS, as it does not require a model of the system. The simulation results illustrated the effectiveness of the RL control method. Next, external disturbances in the form of wind disturbances were used to test the controller's effectiveness compared to conventional PID controllers. Lastly, experiments on a laboratory setup were carried out to confirm the controller's effectiveness in real-world applications.