Double Distillation Network for Multi-Agent Reinforcement Learning

TL;DR

This paper introduces the Double Distillation Network (DDN), a novel approach in multi-agent reinforcement learning that enhances coordination and exploration under partial observability through dual distillation modules.

Contribution

The paper proposes a new DDN framework with external and internal distillation modules to improve collaborative policies and exploration in multi-agent RL.

Findings

Significant performance improvements across multiple scenarios

Enhanced coordination and exploration capabilities

Effective handling of partial observability

Abstract

Multi-agent reinforcement learning typically employs a centralized training-decentralized execution (CTDE) framework to alleviate the non-stationarity in environment. However, the partial observability during execution may lead to cumulative gap errors gathered by agents, impairing the training of effective collaborative policies. To overcome this challenge, we introduce the Double Distillation Network (DDN), which incorporates two distillation modules aimed at enhancing robust coordination and facilitating the collaboration process under constrained information. The external distillation module uses a global guiding network and a local policy network, employing distillation to reconcile the gap between global training and local execution. In addition, the internal distillation module introduces intrinsic rewards, drawn from state information, to enhance the exploration capabilities of agents. Extensive experiments demonstrate that DDN significantly improves performance across multiple scenarios.