Stabilizing Neural Control Using Self-Learned Almost Lyapunov Critics

TL;DR

This paper introduces a novel model-free reinforcement learning approach that learns neural control policies and Lyapunov critics to guarantee stability in nonlinear robotic systems.

Contribution

It develops sample-based methods using Almost Lyapunov functions to estimate stability regions, improving neural controller stability without explicit system models.

Findings

Enhanced stability of neural controllers for automobiles and quadrotors.

Successful estimation of regions of attraction and invariance properties.

Applicable to various nonlinear control systems.

Abstract

The lack of stability guarantee restricts the practical use of learning-based methods in core control problems in robotics. We develop new methods for learning neural control policies and neural Lyapunov critic functions in the model-free reinforcement learning (RL) setting. We use sample-based approaches and the Almost Lyapunov function conditions to estimate the region of attraction and invariance properties through the learned Lyapunov critic functions. The methods enhance stability of neural controllers for various nonlinear systems including automobile and quadrotor control.