Data-Driven Self-Triggered Control for Networked Motor Control Systems Using RNNs and Pre-Training: A Hierarchical Reinforcement Learning Framework

TL;DR

This paper presents a new control system for motor networks using reinforcement learning with pre-trained recurrent neural networks to improve efficiency and accuracy.

Contribution

A novel hierarchical reinforcement learning framework with pre-training for self-triggered control in networked motor systems is introduced.

Findings

The hierarchical policy structure reduces exploration space and improves learning efficiency.

Recurrent neural networks enhance the critic's ability to capture temporal cost dependencies.

Pre-training methods further improve the performance of control policy networks.

Abstract

This paper introduces a novel data-driven self-triggered control approach based on a hierarchical reinforcement learning framework in networked motor control systems. This approach divides the self-triggered control policy into higher and lower layers, with the higher-level policy guiding the lower-level policy in decision-making, thereby reducing the exploration space of the lower-level policy and improving the efficiency of the learning process. The data-driven framework integrates with the dual-actor critic algorithm, using two interconnected neural networks to approximate the hierarchical policies. In this framework, we use recurrent neural networks as the network architecture for the critic, utilizing the temporal dynamics of recurrent neural networks to better capture the dependencies between costs, thus enhancing the critic network’s efficiency and accuracy in approximating the multi-time cumulative cost function. Additionally, we have developed a pre-training method for the control policy networks to further improve learning efficiency. The effectiveness of our proposed method is validated through a series of numerical simulations.

Figures