Learning Multi-Agent Loco-Manipulation for Long-Horizon Quadrupedal Pushing

TL;DR

This paper introduces a hierarchical multi-agent reinforcement learning framework enabling quadrupedal robots to perform long-horizon obstacle-aware pushing tasks, significantly improving success rates and efficiency over baselines in simulation and real-world tests.

Contribution

The paper presents a novel hierarchical multi-agent RL approach combining planning and decentralized policies for complex manipulation by quadrupedal robots.

Findings

36.0% higher success rates than baselines

24.5% reduction in completion time

Successful real-world long-horizon pushing tasks

Abstract

Recently, quadrupedal locomotion has achieved significant success, but their manipulation capabilities, particularly in handling large objects, remain limited, restricting their usefulness in demanding real-world applications such as search and rescue, construction, industrial automation, and room organization. This paper tackles the task of obstacle-aware, long-horizon pushing by multiple quadrupedal robots. We propose a hierarchical multi-agent reinforcement learning framework with three levels of control. The high-level controller integrates an RRT planner and a centralized adaptive policy to generate subgoals, while the mid-level controller uses a decentralized goal-conditioned policy to guide the robots toward these sub-goals. A pre-trained low-level locomotion policy executes the movement commands. We evaluate our method against several baselines in simulation, demonstrating significant improvements over baseline approaches, with 36.0% higher success rates and 24.5% reduction in completion time than the best baseline. Our framework successfully enables long-horizon, obstacle-aware manipulation tasks like Push-Cuboid and Push-T on Go1 robots in the real world.