DMWM: Dual-Mind World Model with Long-Term Imagination

TL;DR

The paper introduces DMWM, a dual-mind world model combining intuitive state transitions with logical reasoning, significantly improving long-term imagination and planning in environment models.

Contribution

It proposes a novel dual-process framework integrating logical reasoning with traditional world models to enhance long-term imagination and planning capabilities.

Findings

Improves logical coherence in long-term predictions.

Enhances sample and data efficiency in environment modeling.

Achieves superior performance on long-horizon planning tasks.

Abstract

Imagination in world models is crucial for enabling agents to learn long-horizon policy in a sample-efficient manner. Existing recurrent state-space model (RSSM)-based world models depend on single-step statistical inference to capture the environment dynamics, and, hence, they are unable to perform long-term imagination tasks due to the accumulation of prediction errors. Inspired by the dual-process theory of human cognition, we propose a novel dual-mind world model (DMWM) framework that integrates logical reasoning to enable imagination with logical consistency. DMWM is composed of two components: an RSSM-based System 1 (RSSM-S1) component that handles state transitions in an intuitive manner and a logic-integrated neural network-based System 2 (LINN-S2) component that guides the imagination process through hierarchical deep logical reasoning. The inter-system feedback mechanism is designed to ensure that the imagination process follows the logical rules of the real environment. The proposed framework is evaluated on benchmark tasks that require long-term planning from the DMControl suite. Extensive experimental results demonstrate that the proposed framework yields significant improvements in terms of logical coherence, trial efficiency, data efficiency and long-term imagination over the state-of-the-art world models.