Learning Latent Part-Whole Hierarchies for Point Clouds

TL;DR

This paper introduces a weakly supervised latent variable model that explicitly learns part-whole hierarchies in 3D point clouds, improving segmentation accuracy at multiple levels.

Contribution

It proposes a novel encoder-decoder framework with discrete latent variables for hierarchical 3D point cloud segmentation, trained with weak supervision.

Findings

Achieves state-of-the-art top-level part segmentation results.

Effectively segments middle-level latent subparts.

Demonstrates robustness with different inference algorithms.

Abstract

Strong evidence suggests that humans perceive the 3D world by parsing visual scenes and objects into part-whole hierarchies. Although deep neural networks have the capability of learning powerful multi-level representations, they can not explicitly model part-whole hierarchies, which limits their expressiveness and interpretability in processing 3D vision data such as point clouds. To this end, we propose an encoder-decoder style latent variable model that explicitly learns the part-whole hierarchies for the multi-level point cloud segmentation. Specifically, the encoder takes a point cloud as input and predicts the per-point latent subpart distribution at the middle level. The decoder takes the latent variable and the feature from the encoder as an input and predicts the per-point part distribution at the top level. During training, only annotated part labels at the top level are provided, thus making the whole framework weakly supervised. We explore two kinds of approximated inference algorithms, i.e., most-probable-latent and Monte Carlo methods, and three stochastic gradient estimations for learning discrete latent variables, i.e., straight-through, REINFORCE, and pathwise estimators. Experimental results on the PartNet dataset show that the proposed method achieves state-of-the-art performance in not only top-level part segmentation but also middle-level latent subpart segmentation.