FORK: A Forward-Looking Actor For Model-Free Reinforcement Learning

TL;DR

This paper introduces FORK, a forward-looking Actor for Actor-Critic algorithms, which enhances performance in continuous control tasks and enables faster learning on complex environments.

Contribution

FORK is a novel Actor design that can be integrated into existing algorithms, significantly improving performance and training speed in continuous reinforcement learning tasks.

Findings

FORK improves performance on six Box2D and MuJoCo environments.

FORK enables solving Bipedal-WalkerHardcore in as little as four hours.

FORK can be integrated into various Actor-Critic algorithms for better results.

Abstract

In this paper, we propose a new type of Actor, named forward-looking Actor or FORK for short, for Actor-Critic algorithms. FORK can be easily integrated into a model-free Actor-Critic algorithm. Our experiments on six Box2D and MuJoCo environments with continuous state and action spaces demonstrate significant performance improvement FORK can bring to the state-of-the-art algorithms. A variation of FORK can further solve Bipedal-WalkerHardcore in as few as four hours using a single GPU.