Digital Twin Synchronization: Bridging the Sim-RL Agent to a Real-Time Robotic Additive Manufacturing Control

TL;DR

This paper presents a novel framework integrating Soft Actor-Critic with digital twins and transfer learning to enable adaptive, real-time control in robotic additive manufacturing, demonstrating rapid convergence and robustness across simulation and real-world tasks.

Contribution

It introduces a hybrid system combining Unity, ROS2, and SAC with transfer learning for adaptive control in industrial robotics, addressing a gap in dynamic process optimization.

Findings

Rapid policy convergence in simulation and real environments

Robust task execution across different control scenarios

Effective digital twin synchronization for real-time control

Abstract

With the rapid development of deep reinforcement learning technology, it gradually demonstrates excellent potential and is becoming the most promising solution in the robotics. However, in the smart manufacturing domain, there is still not too much research involved in dynamic adaptive control mechanisms optimizing complex processes. This research advances the integration of Soft Actor-Critic (SAC) with digital twins for industrial robotics applications, providing a framework for enhanced adaptive real-time control for smart additive manufacturing processing. The system architecture combines Unity's simulation environment with ROS2 for seamless digital twin synchronization, while leveraging transfer learning to efficiently adapt trained models across tasks. We demonstrate our methodology using a Viper X300s robot arm with the proposed hierarchical reward structure to address the common reinforcement learning challenges in two distinct control scenarios. The results show rapid policy convergence and robust task execution in both simulated and physical environments demonstrating the effectiveness of our approach.