Dreamer-CDP: Improving Reconstruction-free World Models Via Continuous Deterministic Representation Prediction

TL;DR

This paper introduces Dreamer-CDP, a new approach that improves reconstruction-free world models in high-dimensional RL tasks by using continuous deterministic representations, matching Dreamer's performance in Crafter.

Contribution

It proposes a JEPA-style predictor on continuous, deterministic representations to enhance reconstruction-free world models, bridging the performance gap with reconstruction-based methods.

Findings

Dreamer-CDP matches Dreamer's performance on Crafter.

The method effectively learns world models without reconstruction objectives.

Reconstruction-free models can perform competitively in high-dimensional RL environments.

Abstract

Model-based reinforcement learning (MBRL) agents operating in high-dimensional observation spaces, such as Dreamer, rely on learning abstract representations for effective planning and control. Existing approaches typically employ reconstruction-based objectives in the observation space, which can render representations sensitive to task-irrelevant details. Recent alternatives trade reconstruction for auxiliary action prediction heads or view augmentation strategies, but perform worse in the Crafter environment than reconstruction-based methods. We close this gap between Dreamer and reconstruction-free models by introducing a JEPA-style predictor defined on continuous, deterministic representations. Our method matches Dreamer's performance on Crafter, demonstrating effective world model learning on this benchmark without reconstruction objectives.