iSDF: Real-Time Neural Signed Distance Fields for Robot Perception

TL;DR

iSDF introduces a real-time neural signed distance field system that improves 3D reconstruction accuracy and detail, enabling better robot perception and planning in indoor environments.

Contribution

The paper presents a novel neural SDF method capable of real-time, adaptive, and denoising 3D reconstruction from depth streams, outperforming voxel-based approaches.

Findings

More accurate reconstructions than prior methods

Better approximation of collision costs and gradients

Effective in real and synthetic indoor environments

Abstract

We present iSDF, a continual learning system for real-time signed distance field (SDF) reconstruction. Given a stream of posed depth images from a moving camera, it trains a randomly initialised neural network to map input 3D coordinate to approximate signed distance. The model is self-supervised by minimising a loss that bounds the predicted signed distance using the distance to the closest sampled point in a batch of query points that are actively sampled. In contrast to prior work based on voxel grids, our neural method is able to provide adaptive levels of detail with plausible filling in of partially observed regions and denoising of observations, all while having a more compact representation. In evaluations against alternative methods on real and synthetic datasets of indoor environments, we find that iSDF produces more accurate reconstructions, and better approximations of collision costs and gradients useful for downstream planners in domains from navigation to manipulation. Code and video results can be found at our project page: https://joeaortiz.github.io/iSDF/ .