Large Scale Robotic Material Handling: Learning, Planning, and Control

TL;DR

This paper presents a comprehensive autonomous system for large-scale material handling that integrates perception, planning, and control, utilizing reinforcement learning modules to optimize efficiency and safety in real-world industrial tasks.

Contribution

It introduces two novel reinforcement learning modules for attack point selection and trajectory control, enabling full-scale automation of material handling operations.

Findings

System achieves high precision and safety in real-world experiments.

Outperforms human operators in efficiency and consistency.

First complete automation of large-scale material handling tasks.

Abstract

Bulk material handling involves the efficient and precise moving of large quantities of materials, a core operation in many industries, including cargo ship unloading, waste sorting, construction, and demolition. These repetitive, labor-intensive, and safety-critical operations are typically performed using large hydraulic material handlers equipped with underactuated grippers. In this work, we present a comprehensive framework for the autonomous execution of large-scale material handling tasks. The system integrates specialized modules for environment perception, pile attack point selection, path planning, and motion control. The main contributions of this work are two reinforcement learning-based modules: an attack point planner that selects optimal grasping locations on the material pile to maximize removal efficiency and minimize the number of scoops, and a robust trajectory following controller that addresses the precision and safety challenges associated with underactuated grippers in movement, while utilizing their free-swinging nature to release material through dynamic throwing. We validate our framework through real-world experiments on a 40 t material handler in a representative worksite, focusing on two key tasks: high-throughput bulk pile management and high-precision truck loading. Comparative evaluations against human operators demonstrate the system's effectiveness in terms of precision, repeatability, and operational safety. To the best of our knowledge, this is the first complete automation of material handling tasks on a full scale.