Meta-Sampler: Almost-Universal yet Task-Oriented Sampling for Point Clouds

TL;DR

This paper introduces Meta-Sampler, a versatile point cloud sampling method that learns to select informative points for various tasks, models, and datasets, reducing overfitting and training costs.

Contribution

The authors propose an almost-universal, task-oriented point cloud sampler trained over multiple models and tasks, enabling rapid adaptation and improved accuracy.

Findings

Training over multiple models improves accuracy over task-specific samplers.

Meta-Sampler can be fine-tuned quickly for different datasets and tasks.

It outperforms traditional and existing learnable sampling methods.

Abstract

Sampling is a key operation in point-cloud task and acts to increase computational efficiency and tractability by discarding redundant points. Universal sampling algorithms (e.g., Farthest Point Sampling) work without modification across different tasks, models, and datasets, but by their very nature are agnostic about the downstream task/model. As such, they have no implicit knowledge about which points would be best to keep and which to reject. Recent work has shown how task-specific point cloud sampling (e.g., SampleNet) can be used to outperform traditional sampling approaches by learning which points are more informative. However, these learnable samplers face two inherent issues: i) overfitting to a model rather than a task, and \ii) requiring training of the sampling network from scratch, in addition to the task network, somewhat countering the original objective of down-sampling to increase efficiency. In this work, we propose an almost-universal sampler, in our quest for a sampler that can learn to preserve the most useful points for a particular task, yet be inexpensive to adapt to different tasks, models, or datasets. We first demonstrate how training over multiple models for the same task (e.g., shape reconstruction) significantly outperforms the vanilla SampleNet in terms of accuracy by not overfitting the sample network to a particular task network. Second, we show how we can train an almost-universal meta-sampler across multiple tasks. This meta-sampler can then be rapidly fine-tuned when applied to different datasets, networks, or even different tasks, thus amortizing the initial cost of training.