Learning Time-optimized Path Tracking with or without Sensory Feedback

TL;DR

This paper introduces a reinforcement learning method enabling robots to follow reference paths in a time-optimized manner, adaptable during motion and capable of utilizing sensory feedback, demonstrated on industrial and humanoid robots.

Contribution

The paper presents a novel learning-based approach for time-optimized path tracking that allows path changes during execution and incorporates sensory feedback, trained entirely in simulation.

Findings

Successful transfer of learned policies from simulation to real robots.

Effective path tracking with adherence to joint limits and time optimization.

Enhanced robustness with sensory feedback for balance and stability.

Abstract

In this paper, we present a learning-based approach that allows a robot to quickly follow a reference path defined in joint space without exceeding limits on the position, velocity, acceleration and jerk of each robot joint. Contrary to offline methods for time-optimal path parameterization, the reference path can be changed during motion execution. In addition, our approach can utilize sensory feedback, for instance, to follow a reference path with a bipedal robot without losing balance. With our method, the robot is controlled by a neural network that is trained via reinforcement learning using data generated by a physics simulator. From a mathematical perspective, the problem of tracking a reference path in a time-optimized manner is formalized as a Markov decision process. Each state includes a fixed number of waypoints specifying the next part of the reference path. The action space is designed in such a way that all resulting motions comply with the specified kinematic joint limits. The reward function finally reflects the trade-off between the execution time, the deviation from the desired reference path and optional additional objectives like balancing. We evaluate our approach with and without additional objectives and show that time-optimized path tracking can be successfully learned for both industrial and humanoid robots. In addition, we demonstrate that networks trained in simulation can be successfully transferred to a real robot.