Maintenance Optimization for Asset Networks with Unknown Degradation Parameters

TL;DR

This paper develops scalable Bayesian and reinforcement learning methods for multi-asset maintenance optimization under unknown, heterogeneous degradation parameters, validated on synthetic and real-world asset networks.

Contribution

It introduces a Bayesian MDP framework and DRL policies that adapt to real-time data, improving maintenance decisions without requiring known degradation parameters.

Findings

DRL methods outperform traditional heuristics in various scenarios.

Policies trained with BMDP remain effective with estimated priors.

Knowledge of true parameters offers marginal cost benefits.

Abstract

We consider the key practical challenge of multi-asset maintenance optimization in settings where degradation parameters are heterogeneous and unknown, and must be inferred from degradation data. To address this, we propose scalable methods suitable for complex asset networks. Degradation is modeled as a stochastic shock process, and real-time data are continuously incorporated into estimation of shock rates and magnitudes via a Bayesian framework. This constitutes a partially observable Markov decision process formulation, from which we analytically derive monotonic policy structures. Moreover, we propose an open-loop feedback approach that enables policies trained via deep reinforcement learning (DRL) in a simulation environment with access to the true parameters to remain effective when deployed with real-time Bayesian point estimates instead. Complementing this, we develop a Bayesian Markov decision process (BMDP) framework wherein the agent maintains and updates posterior distributions during deployment. This formulation captures the evolution of parameter uncertainty over time, thereby facilitating the training of scalable DRL-based policies that adapt as additional data become available. We validate our approach through experiments on synthetic asset networks and a real-world case involving interventional X-ray system filaments. We find that the proposed DRL methods consistently outperform traditional heuristics across various scenarios. The policies trained for the BMDP perform well even when priors must be estimated from historical data, and remain effective in networks with high asset heterogeneity. Knowledge of true degradation parameters yields only marginal cost benefits, underscoring the ability of our approach to make effective decisions under limited information on degradation processes.