Domain Generalization for 6D Pose Estimation Through NeRF-based Image Synthesis

TL;DR

This paper presents a NeRF-based augmentation technique that enhances training data diversity for 6D pose estimation, significantly improving generalization across different domains, especially in spacecraft applications.

Contribution

Introduces a NeRF-based image synthesis method for data augmentation to improve 6D pose estimation generalization.

Findings

Reduces pose estimation error by 50% on SPEED+ dataset.

Enables synthesis of images with unseen viewpoints, varied illumination, and randomized textures.

Significantly improves generalization in spacecraft pose estimation.

Abstract

This work introduces a novel augmentation method that increases the diversity of a train set to improve the generalization abilities of a 6D pose estimation network. For this purpose, a Neural Radiance Field is trained from synthetic images and exploited to generate an augmented set. Our method enriches the initial set by enabling the synthesis of images with (i) unseen viewpoints, (ii) rich illumination conditions through appearance extrapolation, and (iii) randomized textures. We validate our augmentation method on the challenging use-case of spacecraft pose estimation and show that it significantly improves the pose estimation generalization capabilities. On the SPEED+ dataset, our method reduces the error on the pose by 50% on both target domains.