Unsupervised Learning of 3D Semantic Keypoints with Mutual Reconstruction

TL;DR

This paper introduces an unsupervised method for extracting 3D semantic keypoints from point clouds by leveraging mutual reconstruction, enabling explicit semantic consistency and improving upon implicit prior approaches.

Contribution

The paper proposes the first mutual reconstruction-based unsupervised approach for explicitly mining semantic 3D keypoints from point clouds.

Findings

The method achieves superior semantic consistency in keypoints compared to state-of-the-art.

Experiments demonstrate effective reconstruction of objects and categories.

The approach outperforms existing methods on various evaluation metrics.

Abstract

Semantic 3D keypoints are category-level semantic consistent points on 3D objects. Detecting 3D semantic keypoints is a foundation for a number of 3D vision tasks but remains challenging, due to the ambiguity of semantic information, especially when the objects are represented by unordered 3D point clouds. Existing unsupervised methods tend to generate category-level keypoints in implicit manners, making it difficult to extract high-level information, such as semantic labels and topology. From a novel mutual reconstruction perspective, we present an unsupervised method to generate consistent semantic keypoints from point clouds explicitly. To achieve this, the proposed model predicts keypoints that not only reconstruct the object itself but also reconstruct other instances in the same category. To the best of our knowledge, the proposed method is the first to mine 3D semantic consistent keypoints from a mutual reconstruction view. Experiments under various evaluation metrics as well as comparisons with the state-of-the-arts demonstrate the efficacy of our new solution to mining semantic consistent keypoints with mutual reconstruction.