BAA-NGP: Bundle-Adjusting Accelerated Neural Graphics Primitives

TL;DR

BAA-NGP significantly accelerates neural graphics primitives for 3D scene reconstruction and pose estimation, achieving 10-20x faster results than existing methods without losing accuracy, enabling real-time robotic perception.

Contribution

The paper introduces BAA-NGP, a novel framework that uses accelerated sampling and hash encoding to speed up neural scene reconstruction and pose refinement.

Findings

Achieves 10-20x speedup over previous methods

Maintains high-quality pose estimation

Enables real-time 3D scene understanding

Abstract

Implicit neural representations have become pivotal in robotic perception, enabling robots to comprehend 3D environments from 2D images. Given a set of camera poses and associated images, the models can be trained to synthesize novel, unseen views. To successfully navigate and interact in dynamic settings, robots require the understanding of their spatial surroundings driven by unassisted reconstruction of 3D scenes and camera poses from real-time video footage. Existing approaches like COLMAP and bundle-adjusting neural radiance field methods take hours to days to process due to the high computational demands of feature matching, dense point sampling, and training of a multi-layer perceptron structure with a large number of parameters. To address these challenges, we propose a framework called bundle-adjusting accelerated neural graphics primitives (BAA-NGP) which leverages accelerated sampling and hash encoding to expedite automatic pose refinement/estimation and 3D scene reconstruction. Experimental results demonstrate 10 to 20 x speed improvement compared to other bundle-adjusting neural radiance field methods without sacrificing the quality of pose estimation. The github repository can be found here https://github.com/IntelLabs/baa-ngp.