Learning the Approach During the Short-loading Cycle Using Reinforcement Learning

TL;DR

This paper explores using reinforcement learning to train an agent in simulation for the short-loading cycle task, successfully transferring the learned navigation behavior to a real vehicle without further training.

Contribution

It demonstrates the feasibility of applying reinforcement learning in simulation for complex industrial tasks and transferring the learned policy directly to real-world vehicles.

Findings

Agent learned effective navigation in simulation

Successful transfer of policy to real vehicle

Reduced need for additional real-world training

Abstract

The short-loading cycle is a repetitive task performed in high quantities, making it a great alternative for automation. In the short-loading cycle, an expert operator navigates towards a pile, fills the bucket with material, navigates to a dump truck, and dumps the material into the tipping body. The operator has to balance the productivity goal while minimising the fuel usage, to maximise the overall efficiency of the cycle. In addition, difficult interactions, such as the tyre-to-surface interaction further complicate the cycle. These types of hard-to-model interactions that can be difficult to address with rule-based systems, together with the efficiency requirements, motivate us to examine the potential of data-driven approaches. In this paper, the possibility of teaching an agent through reinforcement learning to approach a dump truck's tipping body and get in position to dump material in the tipping body is examined. The agent is trained in a 3D simulated environment to perform a simplified navigation task. The trained agent is directly transferred to a real vehicle, to perform the same task, with no additional training. The results indicate that the agent can successfully learn to navigate towards the dump truck with a limited amount of control signals in simulation and when transferred to a real vehicle, exhibits the correct behaviour.