Focusing Influence Mechanism for Multi-Agent Reinforcement Learning

TL;DR

The paper introduces Focusing Influence Mechanism (FIM), a novel framework that enhances coordination and exploration in multi-agent reinforcement learning with sparse rewards by focusing agents' influence on under-explored states.

Contribution

FIM is a new influence mechanism that uses entropy and eligibility traces to promote coordinated exploration and persistent influence among agents in MARL.

Findings

FIM improves cooperative performance across multiple MARL benchmarks.

FIM facilitates more efficient exploration in sparse reward settings.

FIM outperforms strong baseline methods in experiments.

Abstract

Cooperative multi-agent reinforcement learning (MARL) under sparse rewards remains fundamentally challenging because agents often fail to concentrate their influence, leading to insufficiently coordinated exploration. To address this, we propose the Focusing Influence Mechanism (FIM), a framework that encourages agents to focus their influence on under-explored parts of the state space through an entropy-based criterion, while leveraging eligibility traces to enable multiple agents to consistently align and sustain their influence on the same parts of the state space when beneficial, thereby promoting coordinated and persistent joint behavior. By emphasizing under-explored regions of the state space, FIM facilitates more efficient and structured exploration even under extremely sparse rewards. Across diverse MARL benchmarks, FIM consistently improves cooperative performance over strong baselines.