Reinforcement learning-based optimised control for tracking of nonlinear systems with adversarial attacks

TL;DR

This paper presents a reinforcement learning-based control method for nonlinear systems that effectively handles adversarial attacks, using neural networks to adaptively solve the Hamilton-Jacobi-Bellman equation in real-time.

Contribution

It introduces a novel RL-based control framework with neural networks for nonlinear systems under adversarial attacks, addressing the HJB equation challenge.

Findings

Effective tracking control demonstrated on a manipulator

Neural networks adaptively solve HJB in real-time

Robustness against actuator and output attacks shown

Abstract

This paper introduces a reinforcement learning-based tracking control approach for a class of nonlinear systems using neural networks. In this approach, adversarial attacks were considered both in the actuator and on the outputs. This approach incorporates a simultaneous tracking and optimization process. It is necessary to be able to solve the Hamilton-Jacobi-Bellman equation (HJB) in order to obtain optimal control input, but this is difficult due to the strong nonlinearity terms in the equation. In order to find the solution to the HJB equation, we used a reinforcement learning approach. In this online adaptive learning approach, three neural networks are simultaneously adapted: the critic neural network, the actor neural network, and the adversary neural network. Ultimately, simulation results are presented to demonstrate the effectiveness of the introduced method on a manipulator.