A reinforcement learning agent for maintenance of deteriorating systems with increasingly imperfect repairs

TL;DR

This paper introduces a reinforcement learning approach using a Double Deep Q-Network to optimize maintenance of deteriorating systems with increasingly imperfect repairs, improving long-term costs without predefined thresholds.

Contribution

It proposes a novel maintenance model with a gamma degradation process and an RL agent that operates in continuous state space and adapts to different scenarios.

Findings

The RL agent effectively learns maintenance policies in various scenarios.

The approach significantly reduces long-term maintenance costs.

It operates without predefined preventive thresholds.

Abstract

Efficient maintenance has always been essential for the successful application of engineering systems. However, the challenges to be overcome in the implementation of Industry 4.0 necessitate new paradigms of maintenance optimization. Machine learning techniques are becoming increasingly used in engineering and maintenance, with reinforcement learning being one of the most promising. In this paper, we propose a gamma degradation process together with a novel maintenance model in which repairs are increasingly imperfect, i.e., the beneficial effect of system repairs decreases as more repairs are performed, reflecting the degradational behavior of real-world systems. To generate maintenance policies for this system, we developed a reinforcement-learning-based agent using a Double Deep Q-Network architecture. This agent presents two important advantages: it works without a predefined preventive threshold, and it can operate in a continuous degradation state space. Our agent learns to behave in different scenarios, showing great flexibility. In addition, we performed an analysis of how changes in the main parameters of the environment affect the maintenance policy proposed by the agent. The proposed approach is demonstrated to be appropriate and to significatively improve long-run cost as compared with other common maintenance strategies.