Contextual Knowledge Sharing in Multi-Agent Reinforcement Learning with Decentralized Communication and Coordination

TL;DR

This paper introduces a novel decentralized multi-agent reinforcement learning framework that combines communication and coordination, enabling agents to share contextually relevant, goal-aware, and time-aware knowledge to improve performance in complex, dynamic environments.

Contribution

It presents an integrated Dec-MARL framework that incorporates goal-awareness and time-awareness into peer-to-peer knowledge sharing, addressing limitations of existing methods.

Findings

Enhanced performance in complex tasks with dynamic obstacles.

Effective knowledge sharing improves agent coordination.

Goal and time-awareness significantly boost learning outcomes.

Abstract

Decentralized Multi-Agent Reinforcement Learning (Dec-MARL) has emerged as a pivotal approach for addressing complex tasks in dynamic environments. Existing Multi-Agent Reinforcement Learning (MARL) methodologies typically assume a shared objective among agents and rely on centralized control. However, many real-world scenarios feature agents with individual goals and limited observability of other agents, complicating coordination and hindering adaptability. Existing Dec-MARL strategies prioritize either communication or coordination, lacking an integrated approach that leverages both. This paper presents a novel Dec-MARL framework that integrates peer-to-peer communication and coordination, incorporating goal-awareness and time-awareness into the agents' knowledge-sharing processes. Our framework equips agents with the ability to (i) share contextually relevant knowledge to assist other agents, and (ii) reason based on information acquired from multiple agents, while considering their own goals and the temporal context of prior knowledge. We evaluate our approach through several complex multi-agent tasks in environments with dynamically appearing obstacles. Our work demonstrates that incorporating goal-aware and time-aware knowledge sharing significantly enhances overall performance.