A reinforcement learning based construction material supply strategy using robotic crane and computer vision for building reconstruction after an earthquake

TL;DR

This paper proposes a reinforcement learning-based strategy using robotic cranes and computer vision to safely and efficiently deliver construction materials for rebuilding after earthquakes, reducing hazards and improving logistics.

Contribution

It introduces a novel RL approach with PPO for 3D lift path planning in complex post-earthquake environments, considering obstacle avoidance.

Findings

RL model considering obstacles improves safety and efficiency

Trained models successfully perform in simulated obstacle scenarios

Proposed method reduces collision risks during material transport

Abstract

After an earthquake, it is particularly important to provide the necessary resources on site because a large number of infrastructures need to be repaired or newly constructed. Due to the complex construction environment after the disaster, there are potential safety hazards for human labors working in this environment. With the advancement of robotic technology and artificial intelligent (AI) algorithms, smart robotic technology is the potential solution to provide construction resources after an earthquake. In this paper, the robotic crane with advanced AI algorithms is proposed to provide resources for infrastructure reconstruction after an earthquake. The proximal policy optimization (PPO), a reinforcement learning (RL) algorithm, is implemented for 3D lift path planning when transporting the construction materials. The state and reward function are designed in detail for RL model training. Two models are trained through a loading task in different environments by using PPO algorithm, one considering the influence of obstacles and the other not considering obstacles. Then, the two trained models are compared and evaluated through an unloading task and a loading task in simulation environments. For each task, two different cases are considered. One is that there is no obstacle between the initial position where the construction material is lifted and the target position, and the other is that there are obstacles between the initial position and the target position. The results show that the model that considering the obstacles during training can generate proper actions for the robotic crane to execute so that the crane can automatically transport the construction materials to the desired location with swing suppression, short time consumption and collision avoidance.