Online Model-Free Reinforcement Learning for the Automatic Control of a Flexible Wing Aircraft

TL;DR

This paper introduces an online, model-free reinforcement learning control method for flexible wing aircraft, demonstrating stability and superior performance in simulations despite nonlinear deformations.

Contribution

It develops a novel adaptive-critic reinforcement learning controller that is guaranteed to converge and handle real-time aerodynamic variations in flexible wing aircraft.

Findings

Controller is asymptotically stable in Lyapunov sense

Demonstrates superior performance in simulations

Effective under different operating conditions

Abstract

The control problem of the flexible wing aircraft is challenging due to the prevailing and high nonlinear deformations in the flexible wing system. This urged for new control mechanisms that are robust to the real-time variations in the wing's aerodynamics. An online control mechanism based on a value iteration reinforcement learning process is developed for flexible wing aerial structures. It employs a model-free control policy framework and a guaranteed convergent adaptive learning architecture to solve the system's Bellman optimality equation. A Riccati equation is derived and shown to be equivalent to solving the underlying Bellman equation. The online reinforcement learning solution is implemented using means of an adaptive-critic mechanism. The controller is proven to be asymptotically stable in the Lyapunov sense. It is assessed through computer simulations and its superior performance is demonstrated on two scenarios under different operating conditions.