Autonomous loading of ore piles with Load-Haul-Dump machines using Deep Reinforcement Learning

TL;DR

This paper introduces a deep reinforcement learning approach for autonomous ore pile loading with Load-Haul-Dump machines, demonstrating effective real-world performance and robustness in simulation and scaled experiments.

Contribution

It develops and evaluates RL-based controllers for LHD loading tasks, trained entirely in simulation, with real-world validation and comparison to heuristics and human control.

Findings

RL controllers achieve 71-94% fill factors

Controllers exhibit less wheel drift than baselines

Effective in real-world scaled experiments

Abstract

This work presents a deep reinforcement learning-based approach to train controllers for the autonomous loading of ore piles with a Load-Haul-Dump (LHD) machine. These controllers must perform a complete loading maneuver, filling the LHD's bucket with material while avoiding wheel drift, dumping material, or getting stuck in the pile. The training process is conducted entirely in simulation, using a simple environment that leverages the Fundamental Equation of Earth-Moving Mechanics so as to achieve a low computational cost. Two different types of policies are trained: one with a hybrid action space and another with a continuous action space. The RL-based policies are evaluated both in simulation and in the real world using a scaled LHD and a scaled muck pile, and their performance is compared to that of a heuristics-based controller and human teleoperation. Additional real-world experiments are performed to assess the robustness of the RL-based policies to measurement errors in the characterization of the piles. Overall, the RL-based controllers show good performance in the real world, achieving fill factors between 71-94%, and less wheel drift than the other baselines during the loading maneuvers. A video showing the training environment and the learned behavior in simulation, as well as some of the performed experiments in the real world, can be found in https://youtu.be/jOpA1rkwhDY.