Model-based Reinforcement Learning for Decentralized Multiagent Rendezvous

TL;DR

This paper introduces hierarchical predictive planning (HPP), a model-based reinforcement learning approach enabling decentralized multiagent rendezvous through learned motion predictions, outperforming baselines in complex environments and transferring from simulation to real-world without fine-tuning.

Contribution

The paper presents HPP, a novel hierarchical predictive planning method that combines self-supervised motion prediction with decentralized decision-making for multiagent rendezvous.

Findings

HPP outperforms baseline methods in complex, unseen environments.

Prediction models transfer successfully from simulation to real-world without fine-tuning.

HPP enables decentralized coordination without explicit communication.

Abstract

Collaboration requires agents to align their goals on the fly. Underlying the human ability to align goals with other agents is their ability to predict the intentions of others and actively update their own plans. We propose hierarchical predictive planning (HPP), a model-based reinforcement learning method for decentralized multiagent rendezvous. Starting with pretrained, single-agent point to point navigation policies and using noisy, high-dimensional sensor inputs like lidar, we first learn via self-supervision motion predictions of all agents on the team. Next, HPP uses the prediction models to propose and evaluate navigation subgoals for completing the rendezvous task without explicit communication among agents. We evaluate HPP in a suite of unseen environments, with increasing complexity and numbers of obstacles. We show that HPP outperforms alternative reinforcement learning, path planning, and heuristic-based baselines on challenging, unseen environments. Experiments in the real world demonstrate successful transfer of the prediction models from sim to real world without any additional fine-tuning. Altogether, HPP removes the need for a centralized operator in multiagent systems by combining model-based RL and inference methods, enabling agents to dynamically align plans.