Joint Space Control via Deep Reinforcement Learning

TL;DR

This paper introduces a deep reinforcement learning-based joint space control method for robot manipulators that simplifies traditional control techniques by learning a direct mapping from task space to joint space, achieving high accuracy.

Contribution

The paper presents a novel, differential-equation-free joint control approach using deep RL, capable of handling redundancy, joint limits, and obstacle avoidance, with successful sim-to-real transfer.

Findings

Achieves sub-centimeter accuracy in simulation and real robot.

Handles redundancy, joint limits, and obstacle avoidance automatically.

Simplifies control design by eliminating differential equations.

Abstract

The dominant way to control a robot manipulator uses hand-crafted differential equations leveraging some form of inverse kinematics / dynamics. We propose a simple, versatile joint-level controller that dispenses with differential equations entirely. A deep neural network, trained via model-free reinforcement learning, is used to map from task space to joint space. Experiments show the method capable of achieving similar error to traditional methods, while greatly simplifying the process by automatically handling redundancy, joint limits, and acceleration / deceleration profiles. The basic technique is extended to avoid obstacles by augmenting the input to the network with information about the nearest obstacles. Results are shown both in simulation and on a real robot via sim-to-real transfer of the learned policy. We show that it is possible to achieve sub-centimeter accuracy, both in simulation and the real world, with a moderate amount of training.