DReg-NeRF: Deep Registration for Neural Radiance Fields

TL;DR

DReg-NeRF introduces a novel method for registering multiple neural radiance fields without human intervention, leveraging feature extraction and transformer architectures to outperform existing point cloud registration techniques.

Contribution

The paper presents a transformer-based approach for NeRF registration that does not require human-annotated keypoints or ground truth overlaps, unlike prior methods.

Findings

Outperforms state-of-the-art point cloud registration methods

Achieves mean RPE of 9.67 degrees and RTE of 0.038 on test set

Uses a new dataset with 1,700+ 3D objects from Objaverse

Abstract

Although Neural Radiance Fields (NeRF) is popular in the computer vision community recently, registering multiple NeRFs has yet to gain much attention. Unlike the existing work, NeRF2NeRF, which is based on traditional optimization methods and needs human annotated keypoints, we propose DReg-NeRF to solve the NeRF registration problem on object-centric scenes without human intervention. After training NeRF models, our DReg-NeRF first extracts features from the occupancy grid in NeRF. Subsequently, our DReg-NeRF utilizes a transformer architecture with self-attention and cross-attention layers to learn the relations between pairwise NeRF blocks. In contrast to state-of-the-art (SOTA) point cloud registration methods, the decoupled correspondences are supervised by surface fields without any ground truth overlapping labels. We construct a novel view synthesis dataset with 1,700+ 3D objects obtained from Objaverse to train our network. When evaluated on the test set, our proposed method beats the SOTA point cloud registration methods by a large margin, with a mean $\text{RPE}=9.67^{\circ}$ and a mean $\text{RTE}=0.038$. Our code is available at https://github.com/AIBluefisher/DReg-NeRF.