Toward Geometric Deep SLAM

TL;DR

This paper introduces a deep learning-based point tracking system for SLAM that uses two neural networks to extract stable points and estimate transformations, achieving high speed and robustness without complex training data.

Contribution

It presents MagicPoint and MagicWarp, two neural networks trained on synthetic data for robust point detection and homography estimation in SLAM, bypassing traditional descriptors.

Findings

MagicPoint outperforms classical detectors under noise.

MagicWarp accurately estimates homographies without local descriptors.

System runs at over 30 FPS on a CPU.

Abstract

We present a point tracking system powered by two deep convolutional neural networks. The first network, MagicPoint, operates on single images and extracts salient 2D points. The extracted points are "SLAM-ready" because they are by design isolated and well-distributed throughout the image. We compare this network against classical point detectors and discover a significant performance gap in the presence of image noise. As transformation estimation is more simple when the detected points are geometrically stable, we designed a second network, MagicWarp, which operates on pairs of point images (outputs of MagicPoint), and estimates the homography that relates the inputs. This transformation engine differs from traditional approaches because it does not use local point descriptors, only point locations. Both networks are trained with simple synthetic data, alleviating the requirement of expensive external camera ground truthing and advanced graphics rendering pipelines. The system is fast and lean, easily running 30+ FPS on a single CPU.