World Modeling for Autonomous Wheel Loaders

TL;DR

This paper introduces a data-driven world modeling approach for autonomous wheel loaders, enabling accurate long-horizon planning of loading tasks through neural network-based predictions of pile state and loading performance.

Contribution

It presents a novel neural network-based world model trained on simulation data for predicting loading outcomes, facilitating autonomous planning in dynamic environments.

Findings

Prediction accuracy of 95% for loading performance

Inference times of 1.2 ms for pile state, 4.5 ms for performance

Feasible long-horizon predictions over 40 actions

Abstract

This paper presents a method for learning world models for wheel loaders performing automatic loading actions on a pile of soil. Data-driven models were learned to output the resulting pile state, loaded mass, time, and work for a single loading cycle given inputs that include a heightmap of the initial pile shape and action parameters for an automatic bucket-filling controller. Long-horizon planning of sequential loading in a dynamically changing environment is thus enabled as repeated model inference. The models, consisting of deep neural networks, were trained on data from 3D multibody dynamics simulation of over 10,000 random loading actions in gravel piles of different shapes. The accuracy and inference time for predicting the loading performance and the resulting pile state were, on average, 95% in 1.2 ms and 97% in 4.5 ms, respectively. Long-horizon predictions were found feasible over 40 sequential loading actions.