Eligibility Propagation to Speed up Time Hopping for Reinforcement Learning

TL;DR

This paper introduces Eligibility Propagation, a method that accelerates Time Hopping in Reinforcement Learning by propagating value information through state transition graphs, significantly speeding up learning in simulations.

Contribution

It presents a novel mechanism that enhances Time Hopping with eligibility-like propagation, improving learning speed in simulated environments.

Findings

Eligibility Propagation accelerates learning by over 3 times in experiments.

It provides a similar functionality to eligibility traces for Time Hopping.

The method is validated on a simulated biped crawling robot.

Abstract

A mechanism called Eligibility Propagation is proposed to speed up the Time Hopping technique used for faster Reinforcement Learning in simulations. Eligibility Propagation provides for Time Hopping similar abilities to what eligibility traces provide for conventional Reinforcement Learning. It propagates values from one state to all of its temporal predecessors using a state transitions graph. Experiments on a simulated biped crawling robot confirm that Eligibility Propagation accelerates the learning process more than 3 times.