Multiagent Reinforcement Learning Enhanced Decision-making of Crew Agents During Floor Construction Process

TL;DR

This paper presents a novel multi-agent reinforcement learning framework, CMDP, for simulating and optimizing decision-making in construction crew activities during floor construction, enhancing planning and conflict reduction.

Contribution

Introduces the Construction Markov Decision Process (CMDP) framework tailored for construction, integrating decision, observation, and policy design for intelligent simulation.

Findings

Effective simulation of construction processes in continuous space.

Improved planning and conflict reduction among crews.

Generation of new construction insights through policy analysis.

Abstract

Fine-grained simulation of floor construction processes is essential for supporting lean management and the integration of information technology. However, existing research does not adequately address the on-site decision-making of constructors in selecting tasks and determining their sequence within the entire construction process. Moreover, decision-making frameworks from computer science and robotics are not directly applicable to construction scenarios. To facilitate intelligent simulation in construction, this study introduces the Construction Markov Decision Process (CMDP). The primary contribution of this CMDP framework lies in its construction knowledge in decision, observation modifications and policy design, enabling agents to perceive the construction state and follow policy guidance to evaluate and reach various range of targets for optimizing the planning of construction activities. The CMDP is developed on the Unity platform, utilizing a two-stage training approach with the multi-agent proximal policy optimization algorithm. A case study demonstrates the effectiveness of this framework: the low-level policy successfully simulates the construction process in continuous space, facilitating policy testing and training focused on reducing conflicts and blockages among crews; and the high-level policy improving the spatio-temporal planning of construction activities, generating construction patterns in distinct phases, leading to the discovery of new construction insights.