# Training in Task Space to Speed Up and Guide Reinforcement Learning

## TL;DR

This paper introduces a task space training approach for reinforcement learning that reduces training time and improves robustness by modeling complex systems with simpler ones, utilizing kinematics explicitly, and learning in Cartesian space.

## Contribution

It proposes a novel method combining task space modeling, explicit kinematics, and Cartesian space learning to accelerate and stabilize RL training for high DOF robotic systems.

## Key findings

- Policies trained in minutes on a laptop
- Enhanced robustness compared to other control methods
- Applicable to systems with base and end effectors

## Abstract

Recent breakthroughs in the reinforcement learning (RL) community have made significant advances towards learning and deploying policies on real world robotic systems. However, even with the current state-of-the-art algorithms and computational resources, these algorithms are still plagued with high sample complexity, and thus long training times, especially for high degree of freedom (DOF) systems. There are also concerns arising from lack of perceived stability or robustness guarantees from emerging policies. This paper aims at mitigating these drawbacks by: (1) modeling a complex, high DOF system with a representative simple one, (2) making explicit use of forward and inverse kinematics without forcing the RL algorithm to "learn" them on its own, and (3) learning locomotion policies in Cartesian space instead of joint space. In this paper these methods are applied to JPL's Robosimian, but can be readily used on any system with a base and end effector(s). These locomotion policies can be produced in just a few minutes, trained on a single laptop. We compare the robustness of the resulting learned policies to those of other control methods. An accompanying video for this paper can be found at https://youtu.be/xDxxSw5ahnc .