Real Time Control of Tandem-Wing Experimental Platform Using Concerto Reinforcement Learning

TL;DR

This paper presents CRL2RT, a reinforcement learning-based control algorithm that significantly improves real-time control performance of a tandem-wing platform, achieving high frequencies and better tracking in complex aerodynamic conditions.

Contribution

The paper introduces CRL2RT, a novel hybrid control algorithm combining classical control with reinforcement learning for high-frequency, real-time control of biomimetic aerial vehicles.

Findings

CRL2RT surpasses 2500 Hz control frequency on standard CPUs.

Enhances tracking performance of classical controllers by 18.3% to 60.7%.

Demonstrates broad applicability in complex aerodynamic control scenarios.

Abstract

This paper introduces the CRL2RT algorithm, an advanced reinforcement learning method aimed at improving the real-time control performance of the Direct-Drive Tandem-Wing Experimental Platform (DDTWEP). Inspired by dragonfly flight, DDTWEP's tandem wing structure causes nonlinear and unsteady aerodynamic interactions, leading to complex load behaviors during pitch, roll, and yaw maneuvers. These complexities challenge stable motion control at high frequencies (2000 Hz). To overcome these issues, we developed the CRL2RT algorithm, which combines classical control elements with reinforcement learning-based controllers using a time-interleaved architecture and a rule-based policy composer. This integration ensures finite-time convergence and single-life adaptability. Experimental results under various conditions, including different flapping frequencies and yaw disturbances, show that CRL2RT achieves a control frequency surpassing 2500 Hz on standard CPUs. Additionally, when integrated with classical controllers like PID, Adaptive PID, and Model Reference Adaptive Control (MRAC), CRL2RT enhances tracking performance by 18.3% to 60.7%. These findings demonstrate CRL2RT's broad applicability and superior performance in complex real-time control scenarios, validating its effectiveness in overcoming existing control strategy limitations and advancing robust, efficient real-time control for biomimetic aerial vehicles.