Enhancing Multi-Robot Perception via Learned Data Association

TL;DR

This paper introduces a neural network architecture for multi-robot perception that enables effective data sharing and improves semantic segmentation in multi-view settings with unregistered and overlapping images.

Contribution

We propose the Multi-Agent Infilling Network, a distributed neural architecture that facilitates uncertainty-aware feature exchange among robots for enhanced perception.

Findings

Improved semantic segmentation accuracy on AirSim dataset.

Effective handling of unregistered, overlapping multi-view data.

Distributed neural architecture enables scalable multi-robot perception.

Abstract

In this paper, we address the multi-robot collaborative perception problem, specifically in the context of multi-view infilling for distributed semantic segmentation. This setting entails several real-world challenges, especially those relating to unregistered multi-agent image data. Solutions must effectively leverage multiple, non-static, and intermittently-overlapping RGB perspectives. To this end, we propose the Multi-Agent Infilling Network: an extensible neural architecture that can be deployed (in a distributed manner) to each agent in a robotic swarm. Specifically, each robot is in charge of locally encoding and decoding visual information, and an extensible neural mechanism allows for an uncertainty-aware and context-based exchange of intermediate features. We demonstrate improved performance on a realistic multi-robot AirSim dataset.