TransDreamerV3: Implanting Transformer In DreamerV3

TL;DR

TransDreamerV3 enhances the DreamerV3 reinforcement learning model by integrating a transformer encoder, leading to improved performance in complex environments like Atari and Crafter, demonstrating the benefits of transformer architectures in world model-based RL.

Contribution

It introduces a novel integration of transformer encoders into DreamerV3, advancing memory and decision-making in reinforcement learning models.

Findings

Improved performance on Atari-Freeway and Crafter tasks.

Demonstrated benefits of transformer integration in world models.

Noted limitations in Minecraft task and training scope.

Abstract

This paper introduces TransDreamerV3, a reinforcement learning model that enhances the DreamerV3 architecture by integrating a transformer encoder. The model is designed to improve memory and decision-making capabilities in complex environments. We conducted experiments on Atari-Boxing, Atari-Freeway, Atari-Pong, and Crafter tasks, where TransDreamerV3 demonstrated improved performance over DreamerV3, particularly in the Atari-Freeway and Crafter tasks. While issues in the Minecraft task and limited training across all tasks were noted, TransDreamerV3 displays advancement in world model-based reinforcement learning, leveraging transformer architectures.