Curriculum-Based Soft Actor-Critic for Multi-Section R2R Tension Control

TL;DR

This paper introduces a curriculum-based Soft Actor-Critic controller for multi-section R2R tension control, demonstrating improved generalization and disturbance handling in manufacturing processes.

Contribution

It presents a novel curriculum training approach for SAC in tension control, enabling robust performance across varying conditions without retuning.

Findings

Achieves accurate tension tracking in nominal conditions.

Handles large disturbances effectively with a single policy.

Outperforms traditional model-based control under uncertainty.

Abstract

Precise tension control in roll-to-roll (R2R) manufacturing is difficult under varying operating conditions and process uncertainty. This paper presents a curriculum-based Soft Actor-Critic (SAC) controller for multi-section R2R tension control. The policy is trained in three phases with progressively wider reference ranges, from 27 to 33 N to the full operating envelope of 20 to 40 N, so it can generalize across nominal and disturbed conditions. On a three-section R2R benchmark, the learned controller achieves accurate tracking in nominal operation and handles large disturbances, including 20 N to 40 N step changes, with a single policy and no scenario-specific retuning. These results indicate that curriculum-trained SAC is a practical alternative to model-based control when system parameters vary and process uncertainty is significant.