Flexible Manufacturing Systems Intralogistics: Dynamic Optimization of AGVs and Tool Sharing Using Coloured-Timed Petri Nets and Actor-Critic RL with Actions Masking

TL;DR

This paper presents a novel integrated approach combining Colored-Timed Petri Nets and actor-critic reinforcement learning with action masking to optimize AGV and tool-sharing operations in flexible manufacturing systems, improving efficiency and scalability.

Contribution

It introduces a new hybrid modeling and learning framework for dynamic FMS scheduling, incorporating action masking and lookahead strategies for better performance on large-scale benchmarks.

Findings

Outperforms traditional methods on large-scale benchmarks in makespan reduction.

Achieves tenfold faster computation times compared to existing approaches.

Matches traditional methods on small instances, demonstrating versatility.

Abstract

Flexible Manufacturing Systems (FMS) are pivotal in optimizing production processes in today's rapidly evolving manufacturing landscape. This paper advances the traditional job shop scheduling problem by incorporating additional complexities through the simultaneous integration of automated guided vehicles (AGVs) and tool-sharing systems. We propose a novel approach that combines Colored-Timed Petri Nets (CTPNs) with actor-critic model-based reinforcement learning (MBRL), effectively addressing the multifaceted challenges associated with FMS. CTPNs provide a formal modeling structure and dynamic action masking, significantly reducing the action search space, while MBRL ensures adaptability to changing environments through the learned policy. Leveraging the advantages of MBRL, we incorporate a lookahead strategy for optimal positioning of AGVs, improving operational efficiency. Our approach was evaluated on small-sized public benchmarks and a newly developed large-scale benchmark inspired by the Taillard benchmark. The results show that our approach matches traditional methods on smaller instances and outperforms them on larger ones in terms of makespan while achieving a tenfold reduction in computation time. To ensure reproducibility, we propose a gym-compatible environment and an instance generator. Additionally, an ablation study evaluates the contribution of each framework component to its overall performance.