Temporal Shift Reinforcement Learning

TL;DR

Temporal Shift Reinforcement Learning (TSRL) introduces a method that jointly learns temporal and spatial features in DRL without extra parameters, outperforming frame stacking and setting new state-of-the-art results in Atari environments.

Contribution

TSRL is a novel technique that integrates temporal learning into DRL models without additional parameters, improving performance over traditional methods.

Findings

TSRL outperforms frame stacking in Atari games.

TSRL achieves state-of-the-art results on one Atari environment.

The method has potential applications in robotics and sequential decision-making.

Abstract

The function approximators employed by traditional image-based Deep Reinforcement Learning (DRL) algorithms usually lack a temporal learning component and instead focus on learning the spatial component. We propose a technique, Temporal Shift Reinforcement Learning (TSRL), wherein both temporal, as well as spatial components are jointly learned. Moreover, TSRL does not require additional parameters to perform temporal learning. We show that TSRL outperforms the commonly used frame stacking heuristic on both of the Atari environments we test on while beating the SOTA for one of them. This investigation has implications in the robotics as well as sequential decision-making domains.