Federated Neural Radiance Fields

TL;DR

This paper introduces the first federated learning algorithm for neural radiance fields, enabling multiple agents to collaboratively model a scene without sharing raw data, using low-rank decomposition for efficient communication.

Contribution

It presents a novel federated learning approach for NeRFs, reducing bandwidth and enhancing privacy by transmitting compressed models instead of raw images.

Findings

First federated NeRF training algorithm proposed

Low-rank decomposition reduces communication bandwidth

Supports privacy-preserving collaborative scene modeling

Abstract

The ability of neural radiance fields or NeRFs to conduct accurate 3D modelling has motivated application of the technique to scene representation. Previous approaches have mainly followed a centralised learning paradigm, which assumes that all training images are available on one compute node for training. In this paper, we consider training NeRFs in a federated manner, whereby multiple compute nodes, each having acquired a distinct set of observations of the overall scene, learn a common NeRF in parallel. This supports the scenario of cooperatively modelling a scene using multiple agents. Our contribution is the first federated learning algorithm for NeRF, which splits the training effort across multiple compute nodes and obviates the need to pool the images at a central node. A technique based on low-rank decomposition of NeRF layers is introduced to reduce bandwidth consumption to transmit the model parameters for aggregation. Transferring compressed models instead of the raw data also contributes to the privacy of the data collecting agents.