Predictive reinforcement learning based adaptive PID controller

TL;DR

This paper introduces a novel adaptive PID controller based on predictive reinforcement learning that improves stability and accuracy in controlling nonlinear and unstable systems by integrating model-based and data-driven methods.

Contribution

The study presents a new PRL-PID controller that combines predictive reinforcement learning with adaptive PID control, enhancing robustness and training efficiency.

Findings

Achieves superior stability and tracking accuracy.

Outperforms existing RL-tuned PID methods.

Maintains robustness across diverse conditions.

Abstract

Purpose: This study aims to address the challenges of controlling unstable and nonlinear systems by proposing an adaptive PID controller based on predictive reinforcement learning (PRL-PID), where the PRL-PID combines the advantages of both data-driven and model-driven approaches. Design/methodology/approach: A predictive reinforcement learning framework is introduced, incorporating action smooth strategy to suppress overshoot and oscillations, and a hierarchical reward function to support training. Findings: Experimental results show that the PRL-PID controller achieves superior stability and tracking accuracy in nonlinear, unstable, and strongly coupled systems, consistently outperforming existing RL-tuned PID methods while maintaining excellent robustness and adaptability across diverse operating conditions. Originality/Value: By adopting predictive learning, the proposed PRL-PID integrates system model priors into data-driven control, enhancing both the control framework's training efficiency and the controller's stability. As a result, PRL-PID provides a balanced blend of model-based and data-driven approaches, delivering robust, high-performance control.