A Biologically-Inspired Dual Stream World Model

TL;DR

This paper introduces the Dual Stream World Model (DSWM), inspired by the hippocampus, which learns from high-dimensional data, generates imagined trajectories, and supports reinforcement learning in novel environments.

Contribution

The paper presents a novel dual-stream architecture that dissociates context and content, enabling rapid learning and hippocampus-like representations for AI world models.

Findings

DSWM outperforms standard models in generating trajectories after one exposure.

Latent representations resemble hippocampal place cells.

Generative model aids policy learning with Dyna-like updates.

Abstract

The medial temporal lobe (MTL), a brain region containing the hippocampus and nearby areas, is hypothesized to be an experience-construction system in mammals, supporting both recall and imagination of temporally-extended sequences of events. Such capabilities are also core to many recently proposed ``world models" in the field of AI research. Taking inspiration from this connection, we propose a novel variant, the Dual Stream World Model (DSWM), which learns from high-dimensional observations and dissociates them into context and content streams. DSWM can reliably generate imagined trajectories in novel 2D environments after only a single exposure, outperforming a standard world model. DSWM also learns latent representations which bear a strong resemblance to place cells found in the hippocampus. We show that this representation is useful as a reinforcement learning basis function, and that the generative model can be used to aid the policy learning process using Dyna-like updates.