CoDreamer: Communication-Based Decentralised World Models

TL;DR

CoDreamer extends the Dreamer algorithm to multi-agent reinforcement learning by integrating Graph Neural Networks for communication, improving sample efficiency and cooperation in complex environments.

Contribution

It introduces a novel multi-agent extension of Dreamer that incorporates communication via Graph Neural Networks for better modeling and cooperation.

Findings

Outperforms baseline methods in multi-agent tasks

Enhances expressive power over naive Dreamer applications

Improves sample efficiency and cooperation

Abstract

Sample efficiency is a critical challenge in reinforcement learning. Model-based RL has emerged as a solution, but its application has largely been confined to single-agent scenarios. In this work, we introduce CoDreamer, an extension of the Dreamer algorithm for multi-agent environments. CoDreamer leverages Graph Neural Networks for a two-level communication system to tackle challenges such as partial observability and inter-agent cooperation. Communication is separately utilised within the learned world models and within the learned policies of each agent to enhance modelling and task-solving. We show that CoDreamer offers greater expressive power than a naive application of Dreamer, and we demonstrate its superiority over baseline methods across various multi-agent environments.