USLC: Universal Self-Learning Control via Physical Performance Policy-Optimization Neural Network

TL;DR

This paper introduces a universal self-learning control framework that uses neural networks and a model-free online framework to adaptively optimize control performance in uncertain nonlinear systems, verified through numerical case studies.

Contribution

It presents a novel neural network-based self-learning control method with a model-free online executor-evaluator framework for real-time adaptation in uncertain nonlinear systems.

Findings

Controller achieves near-human performance levels.

Framework demonstrates stability in Lipschitz continuous systems.

Numerical simulations confirm effectiveness across different systems.

Abstract

This study addresses the challenge of achieving real-time Universal Self-Learning Control (USLC) in nonlinear dynamic systems with uncertain models. The proposed control method incorporates a Universal Self-Learning module, which introduces a model-free online executor-evaluator framework to enable controller adaptation in the presence of unknown disturbances. By leveraging a neural network model trained on historical system performance data, the controller can autonomously learn to approximate optimal performance during each learning cycle. Consequently, the controller's structural parameters are incrementally adjusted to achieve a performance threshold comparable to human-level performance. Utilizing nonlinear system stability theory, specifically in the context of three-dimensional manifold space, we demonstrate the stability of USLC in Lipschitz continuous systems. We illustrate the USLC framework numerically with two case studies: a low-order circuit system and a high-order morphing fixed-wing attitude control system. The simulation results verify the effectiveness and universality of the proposed method.