Task Space Control of Hydraulic Construction Machines using Reinforcement Learning

TL;DR

This paper presents a reinforcement learning-based framework for task space control of hydraulic construction machines, using a data-driven actuator model to improve precision and ease of control in complex, real-world operations.

Contribution

It introduces a novel data-driven actuator model integrated with reinforcement learning for effective task space control of hydraulic machines, overcoming the need for explicit dynamic models.

Findings

The framework outperforms traditional Jacobian-based methods.

Experimental validation with Brokk 170 demonstrates improved control accuracy.

The approach enables intuitive and precise machine maneuvering.

Abstract

Teleoperation is vital in the construction industry, allowing safe machine manipulation from a distance. However, controlling machines at a joint level requires extensive training due to their complex degrees of freedom. Task space control offers intuitive maneuvering, but precise control often requires dynamic models, posing challenges for hydraulic machines. To address this, we use a data-driven actuator model to capture machine dynamics in real-world operations. By integrating this model into simulation and reinforcement learning, an optimal control policy for task space control is obtained. Experiments with Brokk 170 validate the framework, comparing it to a well-known Jacobian-based approach.