Robust Optimal Safe and Stability Guaranteeing Reinforcement Learning Control for Quadcopter

TL;DR

This paper presents a novel neural network-based control method for quadcopters that guarantees stability and safety by using Lyapunov functions and sector bounds, improving reliability in critical UAV applications.

Contribution

It extends the ROSS-GT method to design NN controllers with formal stability guarantees for quadcopters, ensuring safety and optimal performance.

Findings

Guarantees closed-loop stability via Lyapunov functions.

Defines invariant safe initial state domains under control.

Optimizes control to minimize tracking error and costs.

Abstract

Recent advances in deep learning have provided new data-driven ways of controller design to replace the traditional manual synthesis and certification approaches. Employing neural network (NN) as controllers however, presents its own challenge: that of certifying stability due to their inherent complex nonlinearity, and while NN controllers have demonstrated high performance in complex systems, they often lack formal stability guarantees. This issue is further accentuated for critical nonlinear applications such as of unmanned aerial vehicles (UAVs), complicating their stability guarantees, whereas a lack of stability assurance raises the risk of critical damage or even complete failure under a loss of control. In this study, we improve a Robust, Optimal, Safe and Stability Guaranteed Training (ROSS-GT) method of [1] to design an NN controller for a quadcopter flight control. The approach ensures closed-loop system stability by finding a Lyapunov function, and providing a safe initial state domain that remains invariant under the control and guarantees stability to an equilibrium within it. Stability guaranteeing constraints are derived from the sector bound of the system nonlinearity and of its parameters and disturbance variations, in the form of a Lipschitz bound for a NN control. The control performance is further optimized by searching over the class of stability-guaranteeing controllers to minimize the reference tracking error and the control costs.