Distributional Reinforcement Learning for Condition-Based Maintenance of Multi-Pump Equipment

TL;DR

This paper introduces a distributional reinforcement learning method using QR-DQN for multi-equipment condition-based maintenance, improving cost efficiency and operational stability in industrial pump systems.

Contribution

It presents a novel QR-DQN based approach for multi-equipment CBM, integrating aging factors and demonstrating superior performance over traditional methods.

Findings

Safety-First strategy achieves ROI of 3.91

System shows 95.66% operational stability

Method outperforms alternatives by 152% in performance

Abstract

Condition-Based Maintenance (CBM) signifies a paradigm shift from reactive to proactive equipment management strategies in modern industrial systems. Conventional time-based maintenance schedules frequently engender superfluous expenditures and unanticipated equipment failures. In contrast, CBM utilizes real-time equipment condition data to enhance maintenance timing and optimize resource allocation. The present paper proposes a novel distributional reinforcement learning approach for multi-equipment CBM using Quantile Regression Deep Q-Networks (QR-DQN) with aging factor integration. The methodology employed in this study encompasses the concurrent administration of multiple pump units through three strategic scenarios. The implementation of safety-first, balanced, and cost-efficient approaches is imperative. Comprehensive experimental validation over 3,000 training episodes demonstrates significant performance improvements across all strategies. The Safety-First strategy demonstrates superior cost efficiency, with a return on investment (ROI) of 3.91, yielding 152\% better performance than alternatives while requiring only 31\% higher investment. The system exhibits 95.66\% operational stability and immediate applicability to industrial environments.