Reducing Redundant Computation in Multi-Agent Coordination through Locally Centralized Execution

TL;DR

This paper presents LCTT, a locally centralized execution framework for multi-agent reinforcement learning that reduces redundant computation by designating leaders and workers, improving efficiency without sacrificing rewards.

Contribution

Introduces LCTT with team-transformer architecture and leadership shift mechanism to minimize redundant computation in decentralized multi-agent systems.

Findings

Reduces redundant computation significantly.

Maintains reward levels comparable to traditional methods.

Achieves faster learning convergence.

Abstract

In multi-agent reinforcement learning, decentralized execution is a common approach, yet it suffers from the redundant computation problem. This occurs when multiple agents redundantly perform the same or similar computation due to overlapping observations. To address this issue, this study introduces a novel method referred to as locally centralized team transformer (LCTT). LCTT establishes a locally centralized execution framework where selected agents serve as leaders, issuing instructions, while the rest agents, designated as workers, act as these instructions without activating their policy networks. For LCTT, we proposed the team-transformer (T-Trans) architecture that allows leaders to provide specific instructions to each worker, and the leadership shift mechanism that allows agents autonomously decide their roles as leaders or workers. Our experimental results demonstrate that the proposed method effectively reduces redundant computation, does not decrease reward levels, and leads to faster learning convergence.