Multiagent Reinforcement Learning with Neighbor Action Estimation

TL;DR

This paper introduces a decentralized multiagent reinforcement learning framework that uses neural networks to estimate neighboring agents' actions, reducing communication needs and improving robustness in real-world robotic tasks.

Contribution

It proposes an action estimation neural network integrated with TD3, enabling agents to infer neighbors' behaviors without explicit communication, scalable to larger systems.

Findings

Improved robustness in robotic manipulation tasks.

Reduced reliance on explicit action sharing.

Enhanced scalability and deployment feasibility.

Abstract

Multiagent reinforcement learning, as a prominent intelligent paradigm, enables collaborative decision-making within complex systems. However, existing approaches often rely on explicit action exchange between agents to evaluate action value functions, which is frequently impractical in real-world engineering environments due to communication constraints, latency, energy consumption, and reliability requirements. From an artificial intelligence perspective, this paper proposes an enhanced multiagent reinforcement learning framework that employs action estimation neural networks to infer agent behaviors. By integrating a lightweight action estimation module, each agent infers neighboring agents' behaviors using only locally observable information, enabling collaborative policy learning without explicit action sharing. This approach is fully compatible with standard TD3 algorithms and scalable to larger multiagent systems. At the engineering application level, this framework has been implemented and validated in dual-arm robotic manipulation tasks: two robotic arms collaboratively lift objects. Experimental results demonstrate that this approach significantly enhances the robustness and deployment feasibility of real-world robotic systems while reducing dependence on information infrastructure. Overall, this research advances the development of decentralized multiagent artificial intelligence systems while enabling AI to operate effectively in dynamic, information-constrained real-world environments.