Multi-Task Multi-Agent Reinforcement Learning via Skill Graphs

TL;DR

This paper introduces a hierarchical multi-task multi-agent reinforcement learning framework using skill graphs, enabling better handling of unrelated tasks and improved knowledge transfer, validated by experiments outperforming existing methods.

Contribution

It presents a novel hierarchical approach with skill graphs for MT-MARL, addressing unrelated tasks and enhancing transfer capabilities, which is a significant advancement over prior methods.

Findings

Outperforms state-of-the-art hierarchical MAPPO algorithms.

Effectively handles unrelated tasks in multi-agent settings.

Enhances knowledge transfer in multi-task reinforcement learning.

Abstract

Multi-task multi-agent reinforcement learning (MT-MARL) has recently gained attention for its potential to enhance MARL's adaptability across multiple tasks. However, it is challenging for existing multi-task learning methods to handle complex problems, as they are unable to handle unrelated tasks and possess limited knowledge transfer capabilities. In this paper, we propose a hierarchical approach that efficiently addresses these challenges. The high-level module utilizes a skill graph, while the low-level module employs a standard MARL algorithm. Our approach offers two contributions. First, we consider the MT-MARL problem in the context of unrelated tasks, expanding the scope of MTRL. Second, the skill graph is used as the upper layer of the standard hierarchical approach, with training independent of the lower layer, effectively handling unrelated tasks and enhancing knowledge transfer capabilities. Extensive experiments are conducted to validate these advantages and demonstrate that the proposed method outperforms the latest hierarchical MAPPO algorithms. Videos and code are available at https://github.com/WindyLab/MT-MARL-SG