SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Data

TL;DR

This paper introduces a novel self-supervised method for estimating 3D object keypoints from noisy, rotated, and decimated point clouds, emphasizing robustness, semantic consistency, and surface proximity without requiring annotations.

Contribution

It presents a new self-supervised training strategy and model architecture that improve keypoints estimation in challenging real-world point cloud data, outperforming existing unsupervised methods.

Findings

Improved keypoints coverage by +9.41%.

Enhanced semantic consistency by +4.66%.

Outperforms state-of-the-art unsupervised approaches.

Abstract

This paper proposes a new method to infer keypoints from arbitrary object categories in practical scenarios where point cloud data (PCD) are noisy, down-sampled and arbitrarily rotated. Our proposed model adheres to the following principles: i) keypoints inference is fully unsupervised (no annotation given), ii) keypoints position error should be low and resilient to PCD perturbations (robustness), iii) keypoints should not change their indexes for the intra-class objects (semantic coherence), iv) keypoints should be close to or proximal to PCD surface (compactness). We achieve these desiderata by proposing a new self-supervised training strategy for keypoints estimation that does not assume any a priori knowledge of the object class, and a model architecture with coupled auxiliary losses that promotes the desired keypoints properties. We compare the keypoints estimated by the proposed approach with those of the state-of-the-art unsupervised approaches. The experiments show that our approach outperforms by estimating keypoints with improved coverage (+9.41%) while being semantically consistent (+4.66%) that best characterizes the object's 3D shape for downstream tasks. Code and data are available at: https://github.com/IITPAVIS/SC3K