A Model-Based Reinforcement Learning Approach for PID Design

TL;DR

This paper introduces a model-based reinforcement learning framework that uses probabilistic inference and divergence measures to automatically tune PID controllers, improving robustness in uncertain systems.

Contribution

It presents a novel method combining PILCO and KLD to derive robust PID parameters from optimal policies, applicable to various control systems.

Findings

Effective PID tuning demonstrated on cart-pole system

Robust performance under disturbances and uncertainties

Generalizable approach blending model-based and model-free methods

Abstract

Proportional-integral-derivative (PID) controller is widely used across various industrial process control applications because of its straightforward implementation. However, it can be challenging to fine-tune the PID parameters in practice to achieve robust performance. The paper proposes a model-based reinforcement learning (RL) framework to design PID controllers leveraging the probabilistic inference for learning control (PILCO) method and Kullback-Leibler divergence (KLD). Since PID controllers have a much more interpretable control structure than a network basis function, an optimal policy given by PILCO is transformed into a set of robust PID tuning parameters for underactuated mechanical systems. The presented method is general and can blend with several model-based and model-free algorithms. The performance of the devised PID controllers is demonstrated with simulation studies for a benchmark cart-pole system under disturbances and system parameter uncertainties.