Learning Point Embeddings from Shape Repositories for Few-Shot Segmentation

TL;DR

This paper introduces a point embedding network that leverages hierarchical and tag metadata from 3D shape repositories to improve semantic segmentation, especially in few-shot learning scenarios.

Contribution

It presents a tree-aware metric-learning approach for learning shape representations that enhance segmentation accuracy with limited training data.

Findings

Reduces segmentation error by 10.2% with 8 examples

Achieves 11.72% error reduction with 120 examples on ShapeNet

Utilizes freely available metadata without extra labeling cost

Abstract

User generated 3D shapes in online repositories contain rich information about surfaces, primitives, and their geometric relations, often arranged in a hierarchy. We present a framework for learning representations of 3D shapes that reflect the information present in this meta data and show that it leads to improved generalization for semantic segmentation tasks. Our approach is a point embedding network that generates a vectorial representation of the 3D points such that it reflects the grouping hierarchy and tag data. The main challenge is that the data is noisy and highly variable. To this end, we present a tree-aware metric-learning approach and demonstrate that such learned embeddings offer excellent transfer to semantic segmentation tasks, especially when training data is limited. Our approach reduces the relative error by $10.2\%$ with $8$ training examples, by $11.72\%$ with $120$ training examples on the ShapeNet semantic segmentation benchmark, in comparison to the network trained from scratch. By utilizing tag data the relative error is reduced by $12.8\%$ with $8$ training examples, in comparison to the network trained from scratch. These improvements come at no additional labeling cost as the meta data is freely available.