Learning to Sample

TL;DR

This paper introduces S-NET, a deep learning approach to learn task-specific sampling of 3D point clouds, outperforming traditional methods like FPS in various applications.

Contribution

It proposes a novel deep network that learns to sample point clouds tailored for specific tasks, improving over traditional sampling techniques.

Findings

S-NET achieves significantly better results than FPS on standard datasets.

Learned sampling improves performance across multiple applications.

The approach is flexible and can be adapted to different downstream tasks.

Abstract

Processing large point clouds is a challenging task. Therefore, the data is often sampled to a size that can be processed more easily. The question is how to sample the data? A popular sampling technique is Farthest Point Sampling (FPS). However, FPS is agnostic to a downstream application (classification, retrieval, etc.). The underlying assumption seems to be that minimizing the farthest point distance, as done by FPS, is a good proxy to other objective functions. We show that it is better to learn how to sample. To do that, we propose a deep network to simplify 3D point clouds. The network, termed S-NET, takes a point cloud and produces a smaller point cloud that is optimized for a particular task. The simplified point cloud is not guaranteed to be a subset of the original point cloud. Therefore, we match it to a subset of the original points in a post-processing step. We contrast our approach with FPS by experimenting on two standard data sets and show significantly better results for a variety of applications. Our code is publicly available at: https://github.com/orendv/learning_to_sample