Using Deep Reinforcement Learning for Zero Defect Smart Forging

TL;DR

This paper presents a digital twin-based deep reinforcement learning framework to optimize heating control in forging lines, reducing defects and automating traditional manual processes in steel forging industries.

Contribution

It introduces a novel digital twin-based DRL approach for automating and optimizing the heating process in forging, addressing challenges of low automation and unforeseen events.

Findings

Significantly reduces temperature unevenness in forging heating processes.

Automates the heating control, decreasing reliance on manual recipes.

Demonstrates effectiveness of DRL models trained with a digital twin environment.

Abstract

Defects during production may lead to material waste, which is a significant challenge for many companies as it reduces revenue and negatively impacts sustainability and the environment. An essential reason for material waste is a low degree of automation, especially in industries that currently have a low degree of digitalization, such as steel forging. Those industries typically rely on heavy and old machinery such as large induction ovens that are mostly controlled manually or using well-known recipes created by experts. However, standard recipes may fail when unforeseen events happen, such as an unplanned stop in production, which may lead to overheating and thus material degradation during the forging process. In this paper, we develop a digital twin-based optimization strategy for the heating process for a forging line to automate the development of an optimal control policy that adjusts the power for the heating coils in an induction oven based on temperature data observed from pyrometers. We design a digital twin-based deep reinforcement learning (DTRL) framework and train two different deep reinforcement learning (DRL) models for the heating phase using a digital twin of the forging line. The twin is based on a simulator that contains a heating transfer and movement model, which is used as an environment for the DRL training. Our evaluation shows that both models significantly reduce the temperature unevenness and can help to automate the traditional heating process.