MASH: Cooperative-Heterogeneous Multi-Agent Reinforcement Learning for Single Humanoid Robot Locomotion

TL;DR

This paper introduces MASH, a cooperative-heterogeneous multi-agent reinforcement learning approach that treats each limb of a humanoid robot as an independent agent to improve locomotion, training efficiency, and coordination.

Contribution

It presents a novel MARL paradigm for single humanoid robots, treating limbs as agents with shared critics, enhancing locomotion and cooperation.

Findings

Faster training convergence compared to single-agent methods

Improved whole-body cooperation in locomotion tasks

Outperforms traditional reinforcement learning approaches

Abstract

This paper proposes a novel method to enhance locomotion for a single humanoid robot through cooperative-heterogeneous multi-agent deep reinforcement learning (MARL). While most existing methods typically employ single-agent reinforcement learning algorithms for a single humanoid robot or MARL algorithms for multi-robot system tasks, we propose a distinct paradigm: applying cooperative-heterogeneous MARL to optimize locomotion for a single humanoid robot. The proposed method, multi-agent reinforcement learning for single humanoid locomotion (MASH), treats each limb (legs and arms) as an independent agent that explores the robot's action space while sharing a global critic for cooperative learning. Experiments demonstrate that MASH accelerates training convergence and improves whole-body cooperation ability, outperforming conventional single-agent reinforcement learning methods. This work advances the integration of MARL into single-humanoid-robot control, offering new insights into efficient locomotion strategies.