A Mini Review on the utilization of Reinforcement Learning with OPC UA

TL;DR

This paper reviews how Reinforcement Learning can be integrated with OPC UA to control industrial processes, highlighting current research, challenges, and the potential for future deployment in industry.

Contribution

It provides a technical overview and a semi-exhaustive literature review on combining RL with OPC UA, identifying key research topics and challenges.

Findings

RL is promising for industrial process control and optimization.

Lack of standardized interfaces hinders real-world deployment.

Research on RL and OPC UA integration is still emerging.

Abstract

Reinforcement Learning (RL) is a powerful machine learning paradigm that has been applied in various fields such as robotics, natural language processing and game playing achieving state-of-the-art results. Targeted to solve sequential decision making problems, it is by design able to learn from experience and therefore adapt to changing dynamic environments. These capabilities make it a prime candidate for controlling and optimizing complex processes in industry. The key to fully exploiting this potential is the seamless integration of RL into existing industrial systems. The industrial communication standard Open Platform Communications UnifiedArchitecture (OPC UA) could bridge this gap. However, since RL and OPC UA are from different fields,there is a need for researchers to bridge the gap between the two technologies. This work serves to bridge this gap by providing a brief technical overview of both technologies and carrying out a semi-exhaustive literature review to gain insights on how RL and OPC UA are applied in combination. With this survey, three main research topics have been identified, following the intersection of RL with OPC UA. The results of the literature review show that RL is a promising technology for the control and optimization of industrial processes, but does not yet have the necessary standardized interfaces to be deployed in real-world scenarios with reasonably low effort.