Federated Neural Radiance Field for Distributed Intelligence

TL;DR

This paper introduces FedNeRF, a federated learning approach for Neural Radiance Fields that enables privacy-preserving novel view synthesis across distributed data sources, addressing data privacy and regulatory challenges.

Contribution

It proposes FedNeRF, integrating federated learning with NeRF, and demonstrates its deployment in a practical FL system with partial client selection, facilitating distributed NeRF applications.

Findings

FedNeRF successfully performs novel view synthesis in distributed data scenarios.

The approach preserves data privacy across multiple data owners.

Experimental results show effective performance with partial client selection.

Abstract

Novel view synthesis (NVS) is an important technology for many AR and VR applications. The recently proposed Neural Radiance Field (NeRF) approach has demonstrated superior performance on NVS tasks, and has been applied to other related fields. However, certain application scenarios with distributed data storage may pose challenges on acquiring training images for the NeRF approach, due to strict regulations and privacy concerns. In order to overcome this challenge, we focus on FedNeRF, a federated learning (FL) based NeRF approach that utilizes images available at different data owners while preserving data privacy. In this paper, we first construct a resource-rich and functionally diverse federated learning testbed. Then, we deploy FedNeRF algorithm in such a practical FL system, and conduct FedNeRF experiments with partial client selection. It is expected that the studies of the FedNeRF approach presented in this paper will be helpful to facilitate future applications of NeRF approach in distributed data storage scenarios.