Learning from demonstrations: An intuitive VR environment for imitation learning of construction robots

TL;DR

This paper introduces a VR-based framework for imitation learning of construction robots, enabling intuitive expert demonstrations and combining BC, GAIL, and PPO to enhance training efficiency and policy performance.

Contribution

The paper presents a novel VR platform for collecting expert demonstrations and a combined imitation learning approach integrating BC, GAIL, and PPO for construction robots.

Findings

Imitation learning with PPO accelerates training.

VR demonstration collection is intuitive and effective.

Combined methods improve policy performance.

Abstract

Construction robots are challenging the traditional paradigm of labor intensive and repetitive construction tasks. Present concerns regarding construction robots are focused on their abilities in performing complex tasks consisting of several subtasks and their adaptability to work in unstructured and dynamic construction environments. Imitation learning (IL) has shown advantages in training a robot to imitate expert actions in complex tasks and the policy thereafter generated by reinforcement learning (RL) is more adaptive in comparison with pre-programmed robots. In this paper, we proposed a framework composed of two modules for imitation learning of construction robots. The first module provides an intuitive expert demonstration collection Virtual Reality (VR) platform where a robot will automatically follow the position, rotation, and actions of the expert's hand in real-time, instead of requiring an expert to control the robot via controllers. The second module provides a template for imitation learning using observations and actions recorded in the first module. In the second module, Behavior Cloning (BC) is utilized for pre-training, Generative Adversarial Imitation Learning (GAIL) and Proximal Policy Optimization (PPO) are combined to achieve a trade-off between the strength of imitation vs. exploration. Results show that imitation learning, especially when combined with PPO, could significantly accelerate training in limited training steps and improve policy performance.