Distributed NeRF Learning for Collaborative Multi-Robot Perception

TL;DR

This paper presents a distributed multi-agent NeRF learning system that enables collaborative environment perception with limited communication, achieving high-quality scene mapping comparable to centralized methods and improving geometric consistency.

Contribution

The paper introduces a novel distributed NeRF learning framework for multi-robot perception, reducing communication overhead and enhancing scene consistency in collaborative mapping.

Findings

Achieves scene reconstruction performance comparable to centralized methods.

Reduces communication overhead by sharing only NeRF models.

Improves geometric consistency and outperforms centralized training in sparse view scenarios.

Abstract

Effective environment perception is crucial for enabling downstream robotic applications. Individual robotic agents often face occlusion and limited visibility issues, whereas multi-agent systems can offer a more comprehensive mapping of the environment, quicker coverage, and increased fault tolerance. In this paper, we propose a collaborative multi-agent perception system where agents collectively learn a neural radiance field (NeRF) from posed RGB images to represent a scene. Each agent processes its local sensory data and shares only its learned NeRF model with other agents, reducing communication overhead. Given NeRF's low memory footprint, this approach is well-suited for robotic systems with limited bandwidth, where transmitting all raw data is impractical. Our distributed learning framework ensures consistency across agents' local NeRF models, enabling convergence to a unified scene representation. We show the effectiveness of our method through an extensive set of experiments on datasets containing challenging real-world scenes, achieving performance comparable to centralized mapping of the environment where data is sent to a central server for processing. Additionally, we find that multi-agent learning provides regularization benefits, improving geometric consistency in scenarios with sparse input views. We show that in such scenarios, multi-agent mapping can even outperform centralized training.