KnowRU: Knowledge Reusing via Knowledge Distillation in Multi-agent Reinforcement Learning

TL;DR

KnowRU introduces a knowledge distillation-based method to efficiently reuse knowledge among agents in multi-agent reinforcement learning, significantly accelerating training and enhancing performance across various scenarios.

Contribution

It presents a simple, adaptable framework for knowledge reuse in MARL that does not require complex design modifications.

Findings

Outperforms existing methods in multiple MARL scenarios

Accelerates training in new tasks

Improves asymptotic performance of agents

Abstract

Recently, deep Reinforcement Learning (RL) algorithms have achieved dramatically progress in the multi-agent area. However, training the increasingly complex tasks would be time-consuming and resources-exhausting. To alleviate this problem, efficient leveraging the historical experience is essential, which is under-explored in previous studies as most of the exiting methods may fail to achieve this goal in a continuously variational system due to their complicated design and environmental dynamics. In this paper, we propose a method, named "KnowRU" for knowledge reusing which can be easily deployed in the majority of the multi-agent reinforcement learning algorithms without complicated hand-coded design. We employ the knowledge distillation paradigm to transfer the knowledge among agents with the goal to accelerate the training phase for new tasks, while improving the asymptotic performance of agents. To empirically demonstrate the robustness and effectiveness of KnowRU, we perform extensive experiments on state-of-the-art multi-agent reinforcement learning (MARL) algorithms on collaborative and competitive scenarios. The results show that KnowRU can outperform the recently reported methods, which emphasizes the importance of the proposed knowledge reusing for MARL.