PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition

TL;DR

This paper introduces PointNetVLAD, a deep learning method combining PointNet and NetVLAD for effective large-scale place recognition using 3D point clouds, with new loss functions and benchmark datasets.

Contribution

It presents a novel end-to-end trainable architecture for point cloud retrieval and introduces 'lazy triplet and quadruplet' loss functions for better descriptor discrimination.

Findings

Demonstrates the effectiveness of PointNetVLAD on new benchmark datasets.

Shows improved global descriptor discriminability for place recognition.

Provides publicly available code and datasets for future research.

Abstract

Unlike its image based counterpart, point cloud based retrieval for place recognition has remained as an unexplored and unsolved problem. This is largely due to the difficulty in extracting local feature descriptors from a point cloud that can subsequently be encoded into a global descriptor for the retrieval task. In this paper, we propose the PointNetVLAD where we leverage on the recent success of deep networks to solve point cloud based retrieval for place recognition. Specifically, our PointNetVLAD is a combination/modification of the existing PointNet and NetVLAD, which allows end-to-end training and inference to extract the global descriptor from a given 3D point cloud. Furthermore, we propose the "lazy triplet and quadruplet" loss functions that can achieve more discriminative and generalizable global descriptors to tackle the retrieval task. We create benchmark datasets for point cloud based retrieval for place recognition, and the experimental results on these datasets show the feasibility of our PointNetVLAD. Our code and the link for the benchmark dataset downloads are available in our project website. http://github.com/mikacuy/pointnetvlad/