Task-Aware Sampling Layer for Point-Wise Analysis

TL;DR

This paper introduces a task-aware, learnable sampling layer for point cloud analysis that improves performance by adaptively selecting points based on task-specific information, outperforming traditional methods like FPS.

Contribution

It proposes a novel joint learning strategy for sampling and task-specific analysis in point clouds, moving beyond fixed sampling methods like FPS.

Findings

Joint learning improves segmentation accuracy.

Adaptive sampling enhances point cloud completion.

Method outperforms FPS in various point-wise tasks.

Abstract

Sampling, grouping, and aggregation are three important components in the multi-scale analysis of point clouds. In this paper, we present a novel data-driven sampler learning strategy for point-wise analysis tasks. Unlike the widely used sampling technique, Farthest Point Sampling (FPS), we propose to learn sampling and downstream applications jointly. Our key insight is that uniform sampling methods like FPS are not always optimal for different tasks: sampling more points around boundary areas can make the point-wise classification easier for segmentation. Towards this end, we propose a novel sampler learning strategy that learns sampling point displacement supervised by task-related ground truth information and can be trained jointly with the underlying tasks. We further demonstrate our methods in various point-wise analysis tasks, including semantic part segmentation, point cloud completion, and keypoint detection. Our experiments show that jointly learning of the sampler and task brings better performance than using FPS in various point-based networks.