A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks

TL;DR

This paper introduces a new multi-agent collaboration task in visually rich environments and proposes novel policies and training methods that significantly improve coordination success rates.

Contribution

It presents SYNC-policies and CORDIAL, novel methods for better multi-agent coordination in complex visual tasks, surpassing existing decentralized approaches.

Findings

Achieved 58% task completion rate with proposed methods.

Outperformed decentralized baselines by 25 percentage points.

Demonstrated effectiveness in a complex furniture-moving task.

Abstract

Autonomous agents must learn to collaborate. It is not scalable to develop a new centralized agent every time a task's difficulty outpaces a single agent's abilities. While multi-agent collaboration research has flourished in gridworld-like environments, relatively little work has considered visually rich domains. Addressing this, we introduce the novel task FurnMove in which agents work together to move a piece of furniture through a living room to a goal. Unlike existing tasks, FurnMove requires agents to coordinate at every timestep. We identify two challenges when training agents to complete FurnMove: existing decentralized action sampling procedures do not permit expressive joint action policies and, in tasks requiring close coordination, the number of failed actions dominates successful actions. To confront these challenges we introduce SYNC-policies (synchronize your actions coherently) and CORDIAL (coordination loss). Using SYNC-policies and CORDIAL, our agents achieve a 58% completion rate on FurnMove, an impressive absolute gain of 25 percentage points over competitive decentralized baselines. Our dataset, code, and pretrained models are available at https://unnat.github.io/cordial-sync .