Autonomous Wheel Loader Navigation Using Goal-Conditioned Actor-Critic MPC

TL;DR

This paper introduces a novel control approach for autonomous wheel loaders that integrates Actor-Critic Reinforcement Learning with Model Predictive Control to achieve efficient goal navigation, validated through simulations and real-world tests.

Contribution

It combines RL-trained critic networks with MPC for improved navigation control, a novel integration enhancing planning capabilities in autonomous wheel loaders.

Findings

MPC with RL critic achieves time-efficient navigation.

The method outperforms traditional trajectory optimization.

Successful real-world deployment demonstrated.

Abstract

This paper proposes a novel control method for an autonomous wheel loader, enabling time-efficient navigation to an arbitrary goal pose. Unlike prior works which combine high-level trajectory planners with Model Predictive Control (MPC), we directly enhance the planning capabilities of MPC by incorporating a cost function derived from Actor-Critic Reinforcement Learning (RL). Specifically, we first train an RL agent to solve the pose reaching task in simulation, then transfer the learned planning knowledge to an MPC by incorporating the trained neural network critic as both the stage and terminal cost. We show through comprehensive simulations that the resulting MPC inherits the time-efficient behavior of the RL agent, generating trajectories that compare favorably against those found using trajectory optimization. We also deploy our method on a real-world wheel loader, where we demonstrate successful navigation in various scenarios.