Reinforcement Learning with Lie Group Orientations for Robotics

TL;DR

This paper introduces a Lie group-based approach for reinforcement learning in robotics, ensuring mathematically correct orientation handling, leading to improved performance across various control tasks.

Contribution

It proposes a simple modification to neural networks that respects Lie group structures for orientations, enhancing learning accuracy and efficiency.

Findings

Significantly better performance than traditional orientation representations.

Effective in direct orientation control and end effector tasks.

Compatible with existing learning libraries.

Abstract

Handling orientations of robots and objects is a crucial aspect of many applications. Yet, ever so often, there is a lack of mathematical correctness when dealing with orientations, especially in learning pipelines involving, for example, artificial neural networks. In this paper, we investigate reinforcement learning with orientations and propose a simple modification of the network's input and output that adheres to the Lie group structure of orientations. As a result, we obtain an easy and efficient implementation that is directly usable with existing learning libraries and achieves significantly better performance than other common orientation representations. We briefly introduce Lie theory specifically for orientations in robotics to motivate and outline our approach. Subsequently, a thorough empirical evaluation of different combinations of orientation representations for states and actions demonstrates the superior performance of our proposed approach in different scenarios, including: direct orientation control, end effector orientation control, and pick-and-place tasks.