CURLing the Dream: Contrastive Representations for World Modeling in Reinforcement Learning

TL;DR

Curled-Dreamer is a new reinforcement learning algorithm that combines contrastive learning with DreamerV3, leading to improved performance, faster learning, and more robust policies in visual tasks.

Contribution

This work introduces Curled-Dreamer, integrating contrastive loss into DreamerV3, which is a novel combination enhancing learning efficiency and robustness in visual reinforcement learning.

Findings

Outperforms state-of-the-art algorithms on DeepMind Control Suite tasks.

Achieves higher mean and median scores across diverse tasks.

Accelerates learning and improves policy robustness.

Abstract

In this work, we present Curled-Dreamer, a novel reinforcement learning algorithm that integrates contrastive learning into the DreamerV3 framework to enhance performance in visual reinforcement learning tasks. By incorporating the contrastive loss from the CURL algorithm and a reconstruction loss from autoencoder, Curled-Dreamer achieves significant improvements in various DeepMind Control Suite tasks. Our extensive experiments demonstrate that Curled-Dreamer consistently outperforms state-of-the-art algorithms, achieving higher mean and median scores across a diverse set of tasks. The results indicate that the proposed approach not only accelerates learning but also enhances the robustness of the learned policies. This work highlights the potential of combining different learning paradigms to achieve superior performance in reinforcement learning applications.