Reinforcement Learning Control of a Forestry Crane Manipulator

TL;DR

This paper explores the application of reinforcement learning to control forestry crane manipulators in simulation, achieving high success rates and energy efficiency improvements in log grasping tasks.

Contribution

It demonstrates the feasibility of using deep reinforcement learning for energy-efficient control of forestry cranes with successful grasping performance.

Findings

97% grasping success rate with learned policies

Energy consumption significantly reduced with energy-optimized reward

Smoother motion profiles observed with energy incentives

Abstract

Forestry machines are heavy vehicles performing complex manipulation tasks in unstructured production forest environments. Together with the complex dynamics of the on-board hydraulically actuated cranes, the rough forest terrains have posed a particular challenge in forestry automation. In this study, the feasibility of applying reinforcement learning control to forestry crane manipulators is investigated in a simulated environment. Our results show that it is possible to learn successful actuator-space control policies for energy efficient log grasping by invoking a simple curriculum in a deep reinforcement learning setup. Given the pose of the selected logs, our best control policy reaches a grasping success rate of 97%. Including an energy-optimization goal in the reward function, the energy consumption is significantly reduced compared to control policies learned without incentive for energy optimization, while the increase in cycle time is marginal. The energy-optimization effects can be observed in the overall smoother motion and acceleration profiles during crane manipulation.