Multi-Year Maintenance Planning for Large-Scale Infrastructure Systems: A Novel Network Deep Q-Learning Approach

TL;DR

This paper introduces a scalable deep reinforcement learning framework for multi-year maintenance planning of large infrastructure networks, effectively handling complexity and budget constraints to improve asset management strategies.

Contribution

It proposes a novel network deep Q-learning approach that decomposes large network MDPs into asset-level problems, enhancing scalability and efficiency in infrastructure maintenance planning.

Findings

Significant efficiency improvements over traditional methods.

Effective handling of large-scale networks with thousands of assets.

Demonstrated success on a pavement network with 68,800 segments.

Abstract

Infrastructure asset management is essential for sustaining the performance of public infrastructure such as road networks, bridges, and utility networks. Traditional maintenance and rehabilitation planning methods often face scalability and computational challenges, particularly for large-scale networks with thousands of assets under budget constraints. This paper presents a novel deep reinforcement learning (DRL) framework that optimizes asset management strategies for large infrastructure networks. By decomposing the network-level Markov Decision Process (MDP) into individual asset-level MDPs while using a unified neural network architecture, the proposed framework reduces computational complexity, improves learning efficiency, and enhances scalability. The framework directly incorporates annual budget constraints through a budget allocation mechanism, ensuring maintenance plans are both optimal and cost-effective. Through a case study on a large-scale pavement network of 68,800 segments, the proposed DRL framework demonstrates significant improvements over traditional methods like Progressive Linear Programming and genetic algorithms, both in efficiency and network performance. This advancement contributes to infrastructure asset management and the broader application of reinforcement learning in complex, large-scale environments.