APSNet: Attention Based Point Cloud Sampling

TL;DR

This paper introduces APSNet, an attention-based neural network that adaptively samples informative points from large 3D point clouds, improving efficiency and performance in downstream tasks like classification and reconstruction.

Contribution

The paper proposes a novel sequential, attention-based sampling method for point clouds, trained with supervised and self-supervised learning, capable of generating variable sample sizes in a task-oriented manner.

Findings

APSNet outperforms existing sampling methods in downstream tasks.

The model effectively learns to select task-relevant points.

Joint training enables flexible sample size generation.

Abstract

Processing large point clouds is a challenging task. Therefore, the data is often downsampled to a smaller size such that it can be stored, transmitted and processed more efficiently without incurring significant performance degradation. Traditional task-agnostic sampling methods, such as farthest point sampling (FPS), do not consider downstream tasks when sampling point clouds, and thus non-informative points to the tasks are often sampled. This paper explores a task-oriented sampling for 3D point clouds, and aims to sample a subset of points that are tailored specifically to a downstream task of interest. Similar to FPS, we assume that point to be sampled next should depend heavily on the points that have already been sampled. We thus formulate point cloud sampling as a sequential generation process, and develop an attention-based point cloud sampling network (APSNet) to tackle this problem. At each time step, APSNet attends to all the points in a cloud by utilizing the history of previously sampled points, and samples the most informative one. Both supervised learning and knowledge distillation-based self-supervised learning of APSNet are proposed. Moreover, joint training of APSNet over multiple sample sizes is investigated, leading to a single APSNet that can generate arbitrary length of samples with prominent performances. Extensive experiments demonstrate the superior performance of APSNet against state-of-the-arts in various downstream tasks, including 3D point cloud classification, reconstruction, and registration.